System Administration

System Administration information

letsencrypt offers free HTTPS certificates with the limitation that you need to set them up to auto renew every ~3 months. Below are some example scripts that last of which should be called from cron.

Settings

DOMAIN=noodle.example.com
EMAIL=me@example.com
PASS=example
DIR=/opt/Noodle/ACME
JKS=/opt/Noodle/noodle.jks

Initialize

. /opt/Noodle/settings.sh
apt install -y certbot
systemctl disable certbot
mkdir $DIR
certbot register --agree-tos --no-eff-email -m "$EMAIL"
certbot certonly --webroot \
	--webroot-path $DIR \
	--cert-name "$DOMAIN" \
	-d "$DOMAIN"
/opt/Noodle/convert.sh

Convert

. /opt/Noodle/settings.sh
P12=$(mktemp)
rm -f "$JKS"
openssl pkcs12 -export \
	-in /etc/letsencrypt/live/"$DOMAIN"/cert.pem \
	-inkey /etc/letsencrypt/live/"$DOMAIN"/privkey.pem \
	-certfile /etc/letsencrypt/live/"$DOMAIN"/fullchain.pem \
	-name "$DOMAIN" \
	-out "$P12" \
	-password "pass:$PASS"
keytool -genkeypair \
	-alias temp \
	-storetype JKS \
	-keystore $JKS \
	-storepass "$PASS" \
	-keypass "$PASS" \
	-dname "CN=temp, OU=temp, O=temp, L=temp, S=temp, C=CA"
keytool -delete \
	-alias temp \
	-keystore $JKS \
	-storepass "$PASS"
keytool -importkeystore \
	-srckeystore "$P12" \
	-srcstoretype pkcs12 \
	-destkeystore $JKS \
	-deststoretype JKS \
	-srcstorepass "$PASS" \
	-deststorepass "$PASS"
rm "$P12"

Renew

. /opt/Noodle/settings.sh
certbot renew --webroot \
	--webroot-path $DIR \
	--cert-name "$DOMAIN" \
	--deploy-hook /opt/Noodle/convert.sh

Currently Noodle must be restarted to reload the keystore.

In 1995 Sun Microsystems created Java. Sun released Java under the GPL in 2006. Oracle Corporation acquired Java with the purchase of Sun in 2010. In 2019 Oracle stopped releasing security updates for LTS releases under a permissive license. OpenJDK is the upstream for all Java builds but does not release builds itself, so there are now many java distributions, some with additional patches. In addition to the Linux distributions (Debian/Ubuntu, Fedora/RedHat/CentOS, Arch/Manjaro, etc) building off OpenJDK for their package managers the following are available;

BuildPermissivePureCommercial Support
AdoptOpenJDK / JClarity / IBMYesOptionalYes
Azul ZuluYesNoYes
Amazon CorrettoYesNoYes
SAPYesNoYes
LibericaYesNoYes
RedHat for WindowsYesNoYes
ojdkbuildYesYesNo
OracleNoNoYes

Windows users can subscribe to releases on github until the windows store catches up with Linux repositories since the year 1998

sudo -i
apt update
apt -y upgrade
apt -y dist-upgrade
apt install -y postgresql-contrib tree p7zip-full sysstat catdoc antiword html2text unrtf libid3-tools ffmpeg openjdk-11-jre
cd /opt
tar -xf ./Noodle.tar.gz
cd ./Noodle
su postgres -c 'cd ~/;psql --file /opt/Noodle/init.sql'
./configure
mv noodle.daemon /etc/init.d/noodle
update-rc.d noodle defaults
service noodle start

  • Drive Space low
    • Add more storage space (a SSD preferably) (can be done without service interruption if you are using a Logical Volume Manager)
  • Out of RAM
    • Add more Memory
    • This warning is not yet supported for MSSQL or remote databases.
  • Index Queue not empty
    • Can be ignored if intermittent small numbers
    • Contact support if large numbers persist
  • HTTP pool too full
    • in multiserver.conf increase HTTP pools like Connection.CM_0.NumThreads to at least double the number of active users. (you may also have a CM_1 if you are using HTTP and HTTPS)
  • DB pool full
    • The default setting of 30 in intranet.conf for DatabaseManager.DB.intranet.Connection.MaxPoolSize is already fairly high so normally this is an indication that the SQL server getting bogged down by slow spinning storage (LVMCache is recommended).
    • Increasing the number without ensuring the DB pool was filled because of volume (not latency) will likely cause out of memory or deadlock errors on the database instead of just slowness.

Overview

IWA allows for SSO using the Operating System users credentials.

Prerequisites

LDAP users have been successfully populated from AD; if not please refer to the AD Configuration Guide.

Quick reference

  1. setspn -U -S HTTP/%noodle.domain.tld@DOMAIN.TLD %service_account
  2. Add Noodle to "Microsoft Internet Explorer > Tools > Internet Options > Security > Local intranet > Sites > Advanced"
  3. Use https://yourNoodle.tld/SPNEGO.po

Supported configurations

Browsers

Microsoft Internet Explorer, Mozilla Firefox, Google Chrome
Note: Microsoft Edge does not support this feature.

Protocols

Kerberos in SPNEGO in GSS-API (RFC 2478)
Note: NTLM, and NegoExts are not supported.

Endpoints

AD, Azure, and F5
Note: Others may work but have not been tested

Server configuration

Define the appropriate SPN for the account which Noodle is using to bind to LDAP.

  • Within the AD Users and Groups snap-in Navigate to Noodle’s service (admin) account and set the follow attribute:
    1. servicePrincipalName = HTTP/%noodle_url
    2. Apply the change
  • This may also be set via command line:
    1. setspn -U -S HTTP/%noodle.domain.tld@DOMAIN.TLD %service_account
  • For Windows server 2008R2 or older command line instead use
    1. setspn -A HTTP/%noodle.domain.tld %service_account

Note:

  • Logins from the AD Kerberos Server itself are not supported because Windows will attempt NTLM.
  • For local installs, non-primary domains can be used with IWA by placing “IgnoreIWADomain = true” into intranet.conf 

Browser configuration

The next step in enabling IWA requires browser configurations to attempt authentication with the Noodle intranet website.

Once the configuration is in place users will need to access Noodle via the SPNEGO.po URL (ie. https://yourNoodle.tld/SPNEGO.po).

The following section of this guide explains this process.

Microsoft Internet Explorer & Google Chrome

Both of these browsers are configured with Microsoft Internet Explorer in "Tools > Internet Options > Security > Local intranet". 

Note:  If a non-default level is in use be sure “Automatic logon only in Intranet zone” is selected in "the Custom Level".

Next we will need to add the URL of your Noodle instance in "Sites > Advanced".

Mozilla Firefox

Enter “about:config” in the address bar, press “Enter”, and click “I Accept the Risk” when prompted.

Next, search for “auth.trusted”, enter your Noodle URL in the attribute entitled “network.negotiate-auth.trusted-uris” and select OK.

At this point IWA should be fully operational for your Noodle Intranet site!

postgresql.conf ships with quite modest defaults so be sure to august it for anything more than light use. We recommend for ~4k users;

  • Available Memory = System Memory - OS (1GB for Windows) - Java (100 to 512)
  • effective_cache_size = Available Memory / 2
  • shared_buffers  = effective_cache_size / 4
  • max_locks_per_transaction = 512
  • max_connections =  60
  • checkpoint_segments 30
  • log_min_duration_statement = 10000
  • log_line_prefix =  ''%m: ''

There is a convenient config creator here.

The first encrypted request after starting Noodle can be 3 orders of magnitude slower than normal due to random entropy pool depletion. If your server suffers from this issue (most EC2 servers) you can verify by checking the pool size:

cd /proc/sys/kernel/random/ && cat poolsize entropy_avail

There are at least 3 solutions to a depleted pool:


1 Hardware

Some hardware includes random generators (like the rdrand CPU flag):

cat /proc/cpuinfo | grep -ic rdrand

You can enable its use with:

Debian/Ubuntu

apt install rng-tools
systemctl start rng-tools.service
systemctl enable rng-tools.service

Fedora/RedHat

yum install rng-tools
systemctl start rngd.service
systemctl enable rngd.service

2 urandom

If you don't have a hardware generator or you don't trust it you can edit /opt/Noodle/multiserver to use:

-Djava.security.egd=file:///dev/urandom

Or edit java.security in $JRE8/lib/security or $JRE9/conf/security to use:

securerandom.source=file:/dev/urandom

3 haveged

For higher throughput and lower CPU usage than urandom haveged can be used but it can still block:

Debian/Ubuntu

apt install haveged
systemctl start haveged.service
systemctl enable haveged.service

Fedora/RedHat

yum install haveged
systemctl start haveged.service
systemctl enable haveged.service

Export your Certificate.

#convert IIS to Tomcat
keytool -importkeystore -srckeystore file.pfx -srcstoretype pkcs12 -destkeystore file.jks -deststoretype JKS
#look up the alias
keytool -storepass 123456 -list -keystore file.jks
# rename the alias for SNI
keytool -storepass 123456 -changealias -keystore file.jks -alias automd5 -destalias domain.tld

Optional:

#import the root from https://certs.godaddy.com/repository
keytool -storepass 123456 -keystore file.jks -importcert -file gdroot-g2.crt -alias root
#import the chain
keytool -storepass 123456 -keystore file.jks -importcert -file gdig2.crt -alias chain