Tools HTTP develop

wrk - HTTP benchmarking

wrk

wrk can be used for simple http benchmarking. It can generate a significant amount of load. You can find it under https://github.com/wg/wrk

A simple call looks like this:

wrk -t12 -c400 -d30s http://www.lvh.me:8080 

This will call the page for 30 seconds with 12 threads and 400 open connections. Afterwords you get statistical output...

Continue...

Docker Monitoring

Docker stats:

docker stats $(docker ps --format=\{\{.Names\}\)

Continue...

ElasticSearch 5.0 fertiggestellt

Elasticsearch, Beats, Logstash und Kibana sind in Version 5.0 veröffentlicht. Die Produkte werden jetzt jeweils in der selben Versionsnummer veröffentlicht.

Es gibt eine neue Scriptsprache "Painless", es wurden Performanceverbesserungen bei bestimmten Suchoperationen und der Indizierung vorgenommen.

Den entsprechenden Blog Eintrag findet ihr hier.


Continue...

Virtualbox slow in new system

I just bought a new computer and used the same home system like before. Virtualbox was now really slow and it used also for small task 100% cpu. I searched through the web without finding a solution.

After a lot of testing i purged virtualbox and vagrant. Additionally i removed the directories .Virtualbox and .vagrant.d in my home...

Continue...

Setup rails server

Create the user and set it up:
useradd --home-dir /var/www -m --shell /bin/bash www
passwd www
groupadd admin
usermod -a -G admin www

Mit Hilfe von visudo wird die Datei /etc/sudoers editiert und folgendes angefügt:

# Members of the admin group may gain root privileges
%admin ALL=(ALL) NOPASSWD: ALL


Continue...

Virtualisation Docker

Setup private docker registry

I am switching more and more to Docker with all my projects. Due to that i setup a private docker registry. I am using puppet for the configuration of the docker registry on my ubuntu server. Basically it is looking like this:

class docker::registry { package {['build-essential', 
		

Continue...

Ruby-on-Rails Development

Welche Programmiersprache ist für mich die richtige?

Mittlerweile gibt es viele Programmiersprachen und so stellt man sich anfangs die Frage welche Programmiersprache man lernen sollte.

Es gibt verschiedene Wege da heran zu gehen. Wenn man weiß was genau man entwickeln möchte, schränkt dies die Auswahl der Programmiersprache natürlich ein. Wenn man Professionel Software entwickerln möchte, kann man sich auch umschauen was am meisten gesucht oder wo am meisten...

Continue...

ALSA Sound

Default sound driver ubuntu

If you have multiple sound cards in your pc  and want to set one as the default card you can do this like this.

With the command:

lsmod | grep snd 

we can find out which sound drivers are loaded. On my pc it is snd_hda_intel and snd_virtuoso.

If you want set the priority of the...

Continue...

Cleanup docker

After some time you collect quite a huge pile of data in docker. Every image and layer is still stored on your harddrive and you might need to clean up to free some disk space.

 

To delete all stopped containers you can use this command

 docker rm $(docker ps -a -q)

To delete all untagged/dangling (none) images use

...

Continue...

docker

Setup private docker registry

I am switching more and more to Docker with all my projects. Due to that i setup a private docker registry. I am using puppet for the configuration of the docker registry on my ubuntu server. Basically it is looking like this:

class docker::registry {
    package {['build-essential', 'python-dev', 'libevent-dev', 'python-pip', 'liblzma-dev']:
        ensure => present
    }

  ...

Continue...

MP3 Playback in Spotify unter (K)Ubuntu 14.04

Unter (K)Ubuntu 14.04 funktioniert das MP3 playback von lokalen Dateien in Spotify nicht mehr out of the box. Um das Playback wieder zu aktivieren muss man folgendes installieren:

1. Herunterladen der libx264 von Saucy http://packages.ubuntu.com/saucy/libx264-123

2. Folgendes PPA für ffpmeg einfügen: https://launchpad.net/~jon-severinsson/+archive/ffmpeg

sudo add-apt-repository ppa:jon-severinsson/ffmpeg

3. ffmpeg installieren

sudo apt-get update
sudo apt-get install...

Continue...

Virtualbox slow in new system

I just bought a new computer and used the same home system like before. Virtualbox was now really slow and it used also for small task 100% cpu. I searched through the web without finding a solution.

After a lot of testing i purged virtualbox and vagrant. Additionally i removed the directories .Virtualbox and .vagrant.d in my home folder.

Now i...

Continue...

Vagrant up is f***ed up

Unbelievable but true, in newer versions of vagrant the provision is only executed once!

Why? You may ask, but i think it is only because some people there not able to make there provisioning scripts idempotent.
For me this is an absolutely unbelievable decision. I was really surprised how many perople liked that. The default should in my opinion be provision...

Continue...

Ruby-on-Rails Development

Mongoid Force Boolean

Unfortunately Mongoid does not check booleans. If you use a checkbox it simply accepts "1", "0" as Strings.

The problem is you can not even query these with true/false. I have found a project on github that Forces booleans to be saved as boolean.

Simply add this include to your models which include booleans:

			Continue...
		
	

Setting up a Server with KVM

I am using kvm on my server to put the applications into different virtual machines. For this i currently try to setup my own tool Mission Control that can be found on github. This Tool is currently under development and is tailored for my special needs.

On all machines i install some default applications like

...
		

Continue...

Ruby-on-Rails Development

Ruby - Code School

Do you want to learn Ruby basics or advanced technics? Then Ruby - Code School might be interesting for you. In these training courses you can learn Ruby Basics, testing, Rspec, patterns and best practices. Currently there are these available courses:

- Learn the basic building blocks of Ruby, all in the browser. Enjoy learning Ruby with video tutorials and...

Continue...

Sun/Oracle Java under Ubuntu

If you want to use the official Oracle Java Version you need to either install it manually or use an unofficial ppa. This is because Ubuntu removed the Oracle/Sun Java from the official repositories. In order to use the ppa you need to issue the following commands:

sudo add-apt-repository ppa:webupd8team/java

sudo apt-get update

sudo apt-get install oracle-java7-installer

 

This will...

Continue...

Allow vagrant to update the nfs export without password

If you want to allow vagrant to change the nfs export without entering an password you can add this to your sudoers file (to edit it just type visudo as root):

# Allow passwordless startup of Vagrant when using NFS.
Cmnd_Alias VAGRANT_EXPORTS_ADD = /usr/bin/su root -c echo '*' >> /etc/exports
Cmnd_Alias VAGRANT_NFSD = /sbin/nfsd restart
Cmnd_Alias VAGRANT_EXPORTS_REMOVE = /usr/bin/sed -e...

Continue...

Ruby-on-Rails Development

Bumbler - Debug ruby startup time

If you wonder why your bundler based application boots so slowly, use bumbler. Bumbler can be added as a gem to your Gemfile: 

 
		

Continue...

Chrome is getting to be the most used browser

I just read on heise.de that the Google Chrome Web-Browser is used by more than 43 percent of the requests. This is a plus of six percent in relation to the beginning of 2013. On the second place is Microsofts Internet Explorer :(. But it lost 5.7 percent in the last six months :). That is very good for every...

Continue...

Ruby-on-Rails Development

Ruby on Rails 4 released

Rails 4 is released! A major feature was to make it really easy to develop modern Webapplications. And this without the use of client side javascript in combination with JSON.

For this the template caching was improved. Turbolinks was added that improves the load time of pages. If available it makes a request via Javascript and replaces the content of the...

Continue...

Setup rails server

Create the user and set it up:
useradd --home-dir /var/www -m --shell /bin/bash www
passwd www
groupadd admin
usermod -a -G admin www

Mit Hilfe von visudo wird die Datei /etc/sudoers editiert und folgendes angefügt:
# Members of the admin group may gain root privileges
%admin ALL=(ALL) NOPASSWD: ALL



Continue...

Installing Torquebox 2.3.1

In order to get the torquebox-server gem installed i needed to increase the memory in order to avoid the 'Java::JavaLang::OutOfMemoryError: Java heap space' error.

jruby -J-Xmx1024m -S gem install torquebox-server -v '2.3.1' --source 'http://torquebox.org/rubygems'

Afterwords everything was fine again.

Continue...

Vagrant startup problem

Vagrant sometimes fails to start the vm with a message like this:
"SSH connection was refused! This usually happens if the VM failed to
boot properly. Some steps to try to fix this: First, try reloading your
VM with `vagrant reload`, since a simple restart sometimes fixes things.
If that doesn't work, destroy your VM and recreate it with a `vagrant destroy`
followed by a...

Continue...

bashrc in Ubuntu 13.04

I was wondering why my bashrc was not executed anymore after the upgrade to Kubuntu 13.04. After some search i found out that the previously needed "--login" param for the bash now seems to ignore the bashrc. After removing this everything worked again.

Continue...

Upgrade to Rails 4

I have tried to convert my first Rails 3 app to Rails 4. I followed roughly the instructions on edge guides.

The most work i had on reworking the attr_accessible that are now removed in rails 4. It is quite easy to migrate, if oyu have e.g. this line in your Model:

attr_accessible :name, :email

And you create your model like
User.new(params[:user])

Then you just...

Continue...

Rails 4 RC

The first release candidate of rails 4 is out!

Rails 4 prefers ruby 2 and ruby 1.9.3 is the minimal requirement. The focus for this release was making it faster. So that you do not need to rely on client side javascript with json to have good performance. It now includes Russian-Doll-Caching and turbolinks.

Also security issues are considered....

Continue...

Vagrant errors in provisioning

If you are using the latest vagrant version from http://www.vagrantup.com/ you do not need virtualbox additionally. If you install it besides the vagrant one it can interfer with vagrant. I needed to uninstall my local vagrant version to get around some errors.

Continue...

Create a Vagrant KVM image

In order to create a KVM vagrant image i used to virt-manager to help me set it up.

Continue...

Steam Sound stutter in Pulseaudio

I had audio stutter in all games i started from steam. Running them without steam works without any problems also playing music, watching videos wasnot a problem.

After some testing i found out that if i restart pulseaudio with "pulseaudio --kill" the sound works for one time. So i found the error message "AL lib: pulseaudio.c:612: Context did not connect: Access denied"...

Continue...

Vagrant KVM

Vagrant packages for your system (Linux, Mac, Windows) can be found under:
http://downloads.vagrantup.com/

First install some dependencies:
sudo apt-get install qemu-kvm libvirt-bin libvirt-dev qemu-system virt-manager

Install the vagrant kvm plugin:
vagrant plugin install vagrant-kvm

You can start it with
vagrant up --provider=kvm

Information about the provider can be found here:
https://github.com/adrahon/vagrant-kvm

For that to work you need a Vagrantfile that already uses version 2. It is not so hard to...

Continue...

Vagrant and Puppet

Short introduction to Vagrant and Puppet

Continue...

Install Google Chrome under (K)Ubuntu 13.04

To install the 64bit Version of Google Chrome under (K)Ubuntu 13.04 you need to download an older Version of libudev which is not in the repositories anymore.

Download libudev0 from launchpad:
http://launchpadlibrarian.net/132294155/libudev0_175-0ubuntu19_amd64.deb

Install Dependencies:
sudo apt-get install libc6 libgconf2-4 libxss1 libcurl3

Download Google Chrome
http://www.google.de/intl/de/chrome/browser/

Install Google Chrome
dpkg -i google-chrome-stable_current_amd64.deb

Continue...

Installing Gnome 3 on (K)Ubuntu

After I heard that Gnome 3 is quite good i wanted to give it a try. I currently use Kubuntu (KDE) so the base system was already installed. And following this article i installed Gnome 3 additionally.

1. sudo add-apt-repository ppa:gnome3-team/gnome3
2. sudo apt-get install ubuntu-gnome-desktop ubuntu-gnome-default-settings
3. sudo apt-get install gnome-documents gnome-boxes

The first step adds a few updated components and is not really necessary....

Continue...

Guard is Your Best Friend

There is a nice guard introduction on tutsplus that shows how to use guard with SASS and coffeescript.

Continue...

Use link_to into Controller in Ruby on Rails 3

You can use the variable
view_context
in a Controller to use the link_to function like this:
view_context.link_to "Link", :root_path

Continue...

GIT Branching Online Training

There is a nice little Online Training for Git branching available. You can interactively learn all the basics and some more advanced features.

Continue...

Postgresql

In Postgres you have to take a deeper look at the user management. You have much more options to secure it for different users than in MySql. But this comes also at a price, i think the easy way of managing access to a role is somehow lost in Postgres. So here are some commands that are important:
The commands under...

Continue...

IP Reputation

If you get a new Domain you should check the reputation first. Your emails may not be delivered to their destination, cause a spammer may have used the ip before. If the IP has a bad reputation spam filters will block it. It can take a very long time before the IP is "clean" again. So the best would be...

Continue...

Capistrano deployment runs test-unit

I had a problem during a capistrano deployment. It executed the unit tests that is should not do and tried to require "RAILS_ENV=test" as file:
file not found: RAILS_ENV=test
The solution was quite simple i just had to add ":require => false" to the test-unit gem:
gem 'test-unit', :require => false

Continue...

Query compatibility problems between Mysql, Postgres, Sqlite

In Mysql the Random  function is called rand. If you want to develop an app that uses mysql and sqlite for development you will have a problem with the naming of the random funcktion:
Postgres, Sqlite: RANDOM()
Mysql: RAND()
 
You can add the alias RANDOM to mysql with this simple sql command:
CREATE FUNCTION random() RETURNS FLOAT NO SQL SQL SECURITY INVOKER RETURN rand();
 
Another...

Continue...

Juju: Services in the cloud

What is Juju?
"With over 100 services ready to deploy, Juju enables you to build entire environments in the cloud with only a few commands on public clouds like Amazon Web Services and HP Cloud, to private clouds built on OpenStack, or raw bare metal via MAAS."
 
So you are able to install services, so called charms, in the cloud. You can...

Continue...

EBooks MongoDB, Redis

You can find two nice ebooks on openmymind.net:
MongoDB
Redis

Continue...

Coding Dojo at Elite Partner

Today i attended on a Coding Dojo at Elite Partner. It was quite interesting to see how other programers look at a problem. For me it was quite hard to realy think in baby steps like it should be in aCoding Dojo. Some of these just seemed so simple.
So what is a Coding Dojo?
On a Coding Dojo...

Continue...

Nginx: Load Balancing

I used pound as a load balancer and currently stumbled upon this article about load balancing with pound. The writer used pound for the site Justin.tv. Pound used about 20% cpu and in spikes up to 80% which is quite a lot. They now switched to nginx and the load dropped to about 3% cpu usage. After reading that i...

Continue...

Postgres pg_large_objects

A nice article can be found here
how to remove large objects in postgres

Continue...

How to Fix MySQL Database (MyISAM / InnoDB)

An article can be found here: http://www.softwareprojects.com/resources/programming/t-how-to-fix-mysql-database-myisam-innodb-1634.html

Continue...

Upgrading to Rails 4

An interesting article about the new security features in Rails 4 and how to get your Rails 3 App ready for it can be found iconclastlabs.

Continue...

Refactoring from good to great

There is a pretty nice talk about refactoring found here.
In the talk he is optimizing the code live.
 
The Topics covered are:
  • The Open-Closed Principle
  • The types of coupling, and their dangers
  • Why composition is so damn great
  • A powerful refactoring that Kent Beck refers to as "deep deep magic"
  • How to destroy conditionals with a NullObject
  • The beauty of...

    Continue...

Torquebox upstart script under ubuntu

In order to let the upstart script for torquebox work on ubuntu you need to change these lines
start on starting network-services
stop on stopping network-services
 
into these
 
start on (started network-interface
          or started network-manager
          or started networking)
stop on (stopped network-interface
          or stopped network-manager
          or stopped networking)
 
The event...

Continue...

DNS in Ubuntu 12.04

In Ubuntu 12.04 the DNS handling ahs changed. The data in /etc/resolv.conf is overwritten at every network change and the resolvconf scripts are used.
To find out some information about the current name servers you can use nm-tool.
If you are using a vagrant box with precise you can add this to your Vagrantfile in order to fix the network problems:
config.vm.customize ["modifyvm",...

Continue...

Hijax: Progressive Enhancement with Ajax

http://domscripting.com/presentations/xtech2006/

Continue...

Migration problems in production mode with JRuby and threadsafe mode

In production mode there is a problem if you have  enabled threadsafe mode.
 
To activate  threadsafe mode you need to uncomment this line in config/environments/production.rb# Enable threaded mode
config.threadsafe! 
After that add this line to fix migration problems in production mode:
 
# Allow rake tasks to autoload models in thread safe mode, more info at http://stackoverflow.com/a/4880253
config.dependency_loading = true if $rails_rake_task
 
 
I also have set...

Continue...

Jruby 1.7

A new Jruby Version 1.7 is released. And now finally the default mode is Ruby 1.9 compatibility. You can read more on the Jruby Site

Continue...

Postgres grant priviliges

Just execute these while connected to the correct database:
GRANT ALL PRIVILEGES ON DATABASE onlinegames_production TO ogi;GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ogi;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO ogi;

Continue...

Zabbix

I currently want to setup a vhost to monitor all servers. It also has to use a zabbix proxy in order to reach everything.
 
Installation on Zabbix Server:
 
sudo apt-get install zabbix-server-mysql
sudo apt-get install zabbix-agent
sudo apt-get install zabbix-frontend-php
Browse to http://yourhostname/zabbix and login as admin with a password of zabbix.
 
On the proxy i installed
zabbix-proxy-sqlite3
and set the database path to /var/lib/zabbix/zabbix.sqlite3
 
On all systems i...

Continue...

Torquebox Capistrano Support

I got the error
 
rake aborted!
no such file to load -- torquebox/capistrano/support
 
while trying to deploy my torquebox app with capistrano. I looked quite some time without finding a solution, but in the chat log under http://transcripts.jboss.org/channel/irc.freenode.org/%23torquebox/2012/%23torquebox.2012-04-17.log i found the simple solution:
 
I just missed this gem
gem "capistrano"
 
cause this was not added automatically as dependency!

Continue...

vmbuilder create an app server

sudo vmbuilder kvm ubuntu --suite quantal --flavour virtual --arch amd64 --libvirt qemu:///system --user 'mike'  --pass 'ubuntu' \
--addpkg openssh-server,vim-nox,acpid,screen,htop,puppet --mirror 'http://mirror.hetzner.de/ubuntu/packages' \
--hostname=app1 --mem '8096' --rootsize '10240' --swapsize '2048'
 
Good??
 
unattended-upgrades
 
Currently there is a bug in quantal (https://bugs.launchpad.net/ubuntu/+source/vm-builder/+bug/1037607), cause of that we need to apply this fix:
https://launchpadlibrarian.net/120169451/usr_share_pyshared_VMBuilder_plugins_ubuntu_dapper.py.diff

Continue...

Capistrano ssh key forwarding

I had quite some problems getting capistrano to check out the repository. After some hours i found this bug  report:
http://jira.codehaus.org/browse/JRUBY-6181
So there is an error with ssh key forwarding in jruby. This is fixed in version JRuby 1.7.0.pre2. So i installed the jruby-head version
In order to get torquebox server installed i needed to increase the max heap memory...

Continue...

Upgrade Ubuntu from a LTS Version to a none LTS Version

Upgrade Ubuntu from a LTS Version to a none LTS Version:
edit the file /etc/update-manager/release-upgrades and set
Prompt=normal
 
Otherwise you can only upgrade to another LTS Release.
Afterwards just do a normal
do-release-upgrade
 
Important: Make sure screen is installed, so you can reconnect to the install if the connection gets lost. The ubuntu update does automatically create a screen session if it is available.

Continue...

(K)ubuntu 12.10 AC3 Pulseaudio

I use the internal digital SPDIF output from my notebook, but it only had stereo output. After some googling the solution was to add a file called /etc/asound.conf and fill it with this content:

pcm.a52 { @args [CARD] @args.CARD { type string } type rate slave { pcm { type a52 card $CARD } rate 48000 #required...
		

Continue...

Capistrano Torquebox

Install Capistrano:
jruby -S gem install jruby-openssl ffi-ncurses capistrano
 
Capify project:
jruby -S capify .
 
Add this to the Capfile:
require 'org.torquebox.capistrano-support'
require 'bundler/capistrano'
 
Important is that the servers are in the ssh allowed list. So check if you can reach the servers with ssh and have accepted the ssh key permanently.

Continue...

Server Security: Disable root login

I would recommened that you do not use the root user to access any server. It is better to disable the root login, cause many automated scripts are trying to access the root user via ssh with some common passwords.
On my server i setup another user which afterwords is allowed to become root, e.g.:
 
useradd --home-dir /home/my_user -m --shell /bin/bash my_user
passwd...

Continue...

ISPConfig

This is what we did to setup ISPConfig on a friends server.
Install vim:
 aptitude install vim-nox
 
Setup correct hostname:
 
echo server1.example.com > /etc/hostname
/etc/init.d/hostname restarthostname
hostname -f
 
NTP:
 
aptitude install ntp ntpdate
 
Mail:
The mail server needs an own subdomain that is not needed as email domain. You can add mail. to your domain, e.g. mail.example.org
 
aptitude install postfix postfix-mysql postfix-doc mysql-client mysql-server libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl getmail4...

Continue...

Rails with MongoDB (Mongoid)

If we want to use Mongoid we do not need ActiveRecord, so we could create an app like this:
 rails new SuperApp --skip-active-record 
 
The following gems are needed
 
# Mongo DB
gem 'mongoid'
gem 'bson_ext'
 
The bson_ext gem is used for faster bson conversion.
Mongoid Documentation

Continue...

About me

I am interested in computers since my 12th birthday or so. I became a C64 on which i played a lot. But i also started programing in Basic and Assembler. It was really facinating so i used quite a lot of my time on it.
A few years later i bought my first PC. It was 486SX with 25Mhz and cost...

Continue...

This page is intended to contain some notes on my server setup, technologies i use or tested and programing related stuff.Maybe this will also help some others.

© Copyright 2017. Mike Petersen Odania IT