System Administration

System Administration information

•

postgresql.conf ships with quite modest defaults so be sure to adjust it for anything more than light use. We recommend for ~4k users;

  • Available Memory = System Memory - OS (1GB for Windows) - Java (100 to 512)
  • effective_cache_size = Available Memory / 2
  • shared_buffers  = effective_cache_size / 4
  • max_locks_per_transaction = 512
  • max_connections =  60
  • checkpoint_segments 30
  • log_min_duration_statement = 10000
  • log_line_prefix =  ''%m: ''

Official documentation here.

There are convenient config creators here and here.

Stop the Noodle service and restart the PostgreSQL service, then start the Noodle service again to apply the changes.

•

The first encrypted request after starting Noodle can be 3 orders of magnitude slower than normal due to random entropy pool depletion. If your server suffers from this issue (most EC2 servers) you can verify by checking the pool size:

cd /proc/sys/kernel/random/ && cat poolsize entropy_avail

There are at least 3 solutions to a depleted pool:

1 Hardware

Some hardware includes random generators (like the rdrand CPU flag):

grep -ic rdrand /proc/cpuinfo

You can enable its use with:

Debian/Ubuntu

apt install rng-tools
systemctl start rng-tools.service
systemctl enable rng-tools.service

Fedora/RedHat

yum install rng-tools
systemctl start rngd.service
systemctl enable rngd.service

2 urandom

If you don't have a hardware generator or you don't trust it you can edit /opt/Noodle/multiserver to use:

-Djava.security.egd=file:///dev/urandom

Or edit java.security in $JRE8/lib/security or $JRE9/conf/security to use:

securerandom.source=file:/dev/urandom

3 haveged

For higher throughput and lower CPU usage than urandom haveged can be used but it can still block:

Debian/Ubuntu

apt install haveged
systemctl start haveged.service
systemctl enable haveged.service

Fedora/RedHat

yum install haveged
systemctl start haveged.service
systemctl enable haveged.service

• 
#Helpers
yum install -y epel-release
yum install -y bc ntp.x86_64 screen tree catdoc id3lib html2text unrtf p7zip-plugins.x86_64  libjpeg-turbo.x86_64 poppler-utils unzip.x86_64 netpbm-progs.x86_64 perl-Image-ExifTool.noarch postgresql-server.x86_64 postgresql-contrib.x86_64 java-11-openjdk.x86_64;
rpm -Uvh http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm;
yum install -y ffmpeg;
newcfg (){
	F="$1"
	chown --reference="$F" "$F".new
	chmod --reference="$F" "$F".new
	mv -f "$F".new "$F"
}

#PostgreSQL
/usr/bin/postgresql-setup initdb
chkconfig  --level 235 postgresql on
TMP=/var/lib/pgsql/data/pg_hba.conf
cp $TMP $TMP.original
cat $TMP | grep -vP "^ *host.*127" > $TMP.new
echo -e "host\tall\tall\t127.0.0.1/32\tpassword" >> $TMP.new
newcfg $TMP
TMP=/var/lib/pgsql/data/postgresql.conf
cp $TMP $TMP.original
MT=$(cat /proc/meminfo | grep MemTotal | perl -pe 's/^[^ ]* *([0-9]+) *kB$/$1\/1000/g');
#java+linux=(512+256)
ECS=$(echo "($MT-(512+256))/2" | bc);
SB=$(echo "($MT-(512+256))/4" | bc);
cat $TMP | perl -pe 's/^#?(effective_cache_size)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = '$ECS'MB$2/g' \
| perl -pe 's/^#?(shared_buffers)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = '$SB'MB$2/g' \
| perl -pe 's/^#?(max_locks_per_transaction)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = 512$2/g' \
| perl -pe 's/^#?(max_connections)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = 60$2/g' \
| perl -pe 's/^#?(checkpoint_segments)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = 30$2/g' \
| perl -pe 's/^#?(log_min_duration_statement)[ \t]*=[ \t]*[^ \t]+([ \t].*)?$/$1 = 30000$2/g' \
| perl -pe 's/^#?(log_line_prefix)[ \t]*=[ \t]*[^#]+(#.*)?$/$1 = '\''%m: '\''\t\t$2/g' \
> $TMP.new
newcfg $TMP
service postgresql start

#Noodle (first take note of your available ram and java location)
tar -zxf ./Noodle.tar.gz
su postgres -c 'cd ~/;/usr/bin/psql -d postgres -U postgres --file /opt/Noodle/init.sql' 
cd ./Noodle
cp noodle.daemon /etc/init.d/noodle
chkconfig --add noodle
chkconfig --levels 235 noodle on
service noodle start

Remember to permit traffic through your remote and local firewalls.

firewall-cmd --add-service=http
firewall-cmd --runtime-to-permanent

Other Linux Installation examples available.

•

(see also latest version howto and version notes)

ssh root@my.server.tld

###########################################################
##postgres
yum install postgresql postgresql-server postgresql-server postgresql-contrib
chkconfig --level 235 postgresql on
service postgresql start
vim /var/lib/pgsql/data/pg_hba.conf
#    127.0.0.1/32 password
vim /var/lib/pgsql/data/postgresql.conf
#    maintenance_work_mem=128000
#    autovacuum = on
service postgresql restart
su postgres
pgsql -U postgres
CREATE LANGAUGE plpgsql;
CREATE DATABASE noodledb WITH ENCODING = 'UNICODE';
create user noodleuser with superuser password 'some long password';
GRANT ALL PRIVILEGES ON DATABASE noodledb to noodleuser;
\q
exit

###########################################################
##java (get an updated url from java.sun.com)
cd /opt
wget 'http://javadl.oracle.com/webapps/download/AutoDL?BundleId=207765' -O jre.tar.gz
tar -zxf jre.tar.gz
rm -f jre.tar.gz

###########################################################
##7z
#    you can use yum-priorities and rpmforge or you can do it from source
yum install p7zip p7zip-plugins
#    you might need to make a link from 7z to 7z[ar] for "which 7z" to work

###########################################################
##Noodle
wget somewhere/Noodle.tar.gz
tar -xvvf ./Noodle.tar.gz
cd ./noodle

Other Linux Installation examples available.

• 
sudo -i
apt update
apt -y upgrade
apt -y dist-upgrade
apt install -y postgresql-contrib tree p7zip-full sysstat catdoc antiword html2text unrtf libid3-tools ffmpeg openjdk-8-jre poppler-utils unzip
cd /opt
tar -xf ./Noodle.tar.gz
cd ./Noodle
su postgres -c 'cd ~/;psql --file /opt/Noodle/init.sql'
mv noodle.daemon /etc/init.d/noodle
update-rc.d noodle defaults
service noodle start

Other Linux Installation examples available.

•

(see also howto get a new version of PGSQL )

sudo -i
apt-get install -y postgresql-9.3 postgresql-contrib-9.3 default-jre p7zip-full screen
cd /opt
tar -xf ./Noodle.tar.gz
cd ./Noodle
echo "host all all 127.0.0.1/32 password" >> /etc/postgresql/9.3/main/pg_hba.conf
service postgres start
su postgres -c 'cd ~/;psql --file /opt/Noodle/init.sql'
mv noodle.daemon /etc/init.d/noodle
service noodle start
update-rc.d noodle defaults

Other Linux Installation examples available.