Ended up moving my server over to an Intel NUC D54250WYK Haswell i5 with attached storage. Ah, the advantages of running a VM. Intel finally released this low-power desktop system and it seems to work great with Esxi 5.5, with the addition of a few drivers.
The problem is described here. However, in fact, the driver the author provided for the e1001 did not work for my system. I used a combination of the 5.5 iso (VMware-VMvisor-Installer-5.5.0-1331820.x86_64.iso), ESXI-customizer 2.7.1, and two drivers, the e1000 (net-e1000e-2.3.2.x86_64.vib) from here and sata-xhci (sata-xahci-1.5-1.x86_64.vib) from here. I've provided a mirror for the drivers and the customerizer on dropbox here.
The ESXI-customizer takes care of most of the nitty gritty, so just follow the instructions there for adding the drivers, one at a time.
VT-d and PCI passthrough seems to work just fine with this hardware.
Tech Notes
Thursday, December 12, 2013
Saturday, June 8, 2013
IRC Web Client
For the web interface, we will install qwebirc and proxy it through Nginx on Ubuntu 12.04 Precise Pangolin. The instructions for qwebirc are kind of in different places, so hopefully this will help someone.
In my case JDK is already installed, so the dependencies I needed where
To apply the patch cd to the source root and do
Set IDENT to a valid user on your LDAP domain. I created an account called "webirc" in ldap-account-manager.
In my case JDK is already installed, so the dependencies I needed where
sudo apt-get install python python-twisted python-twisted-bin \Download the source somewhere
python-twisted-core python-twisted-runner python-twisted-names \
python-twisted-mail python-twisted-words python-twisted-web \
python-zope.interface python-openss mercurial
hg clone http://hg.qwebirc.org/qwebirc qwebircI decided to switch to their stable branch
hg up -C stableOk, in my case, I needed to apply two patches for SSL and server authentication. For the authentication I applied the patch from here, direct link to the patch here. For the SSL support I applied the patch from anacart's post in this thread, direct link to the patch here.
To apply the patch cd to the source root and do
patch -p1 < patch.diffOnce your done, put the qwebirc folder somewhere permanent, like /usr/local/qwebirc, or /usr/share/qwebirc, and make a copy of the config file.
cp config.py.example config.pyEdit config.py. Change IRCPORT and SSL port to match the client port of your IRC server.
Set IDENT to a valid user on your LDAP domain. I created an account called "webirc" in ldap-account-manager.
IDENT = "webirc"Set the NETWORK to the IRC network name, specified in the inspircd.conf
NETWORK_NAME = "IRCNet"I wasn't sure what to see the URLs to, but here's how mine is set. Set REALNAME to the server address.
REALNAME = "https://www.domain.com/webirc"Set BASE_URL to the local address, i don't think this is right, needs checking.
BASE_URL = "http://localhost:9090"For the Nginx proxy, set the following.
FORWARDED_FOR_HEADER="x-forwarded-for"Finally compile qwebirc.
FORWARDED_FOR_IPS=["127.0.0.1"]
python compile.pyAnd test it
python run.pyYou should be able to browse to http://localhost:9090/
Lastly, create a file to launch qwebirc as a service. If qwebirc crashes, this script will not restart the process, it needs some tweaking. Edit /etc/init/qwebirc.conf
# qwebirc - qwebirc job file
start on runlevel [2345]stop on runlevel [016]
chdir /usr/local/qwebirc
export fork
exec /usr/local/qwebirc/run.py
If all goes well, it should start on reboot, or by running
sudo service qwebirc startNow to tweak Nginx. I just had to add the following to /etc/nginx/sites-enabled/default
location ^~ /webirc/ {Restart nginx
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://127.0.0.1:9090/;
}
sudo service nginx restartAnd browse to https://www.domain.com/webirc
IRC Server
Decided to setup a little IRC server using Inspircd. The plan was as follows. Use LDAP authentication for users, SSL encryption using my domain certs, a web-based UI proxied through NGINX https and lastly, federated with a friend's IRC server. Ubuntu 12.04 comes with an old version of Inspircd (1.1.2?), doesn't work well with federation and doesn't come with the LDAP module. I downloaded the latest version of Inspircd that comes with 12.10 Ubuntu from here.
The above link also shows any dependencies you may need to install separately. I was missing a few, solved with...
The main config file to edit is /etc/inspircd/inspircd.conf You'll want to configure this to setup some basic info, like change the <bind> tag to listen on an ip other than "127.0.0.1". "" will default to all network interfaces, and then set the port to listen to.
At this point, you can run the Inspircd daemon
First you need to tell Inspircd to load the gnutls module and point to your certs, by editing /etc/inspircd/inspircd.conf and adding:
The above link also shows any dependencies you may need to install separately. I was missing a few, solved with...
sudo apt-get install libtre5 libpq5 libmysqlclient18then installed Inspircd
sudo dpkg -i inspircd_2.0.5-1_amd64.debFirst you need to edit /etc/default/inspircd and change the '0' to '1'
The main config file to edit is /etc/inspircd/inspircd.conf You'll want to configure this to setup some basic info, like change the <bind> tag to listen on an ip other than "127.0.0.1". "" will default to all network interfaces, and then set the port to listen to.
At this point, you can run the Inspircd daemon
sudo service inspircd startNext will secure the chat client port to use the SSL cert for the server. I store the SSL certs with my nginx server in /etc/nginx/certs.
First you need to tell Inspircd to load the gnutls module and point to your certs, by editing /etc/inspircd/inspircd.conf and adding:
<module name="m_ssl_gnutls.so">If you want, you can create a self-signed cert, and use that, but clients will need to be told to ignore invalid certs.
<gnutls certfile="/etc/nginx/certs/server.crt" keyfile="/etc/nginx/certs/server.key">
Next, change your client's bind tag to something like:
<bind address="" port="5309" type="clients" ssl="gnutls">To add LDAP authentication, you need to load the ldapauth module and point to your ldap server, by editing /etc/inspircd/inspircd.conf and adding:
<module name="m_ldapauth.so">To connect this server to another server, you need to <bind> a port as type server,
<ldapauth baserdn="ou=People,dc=domain,dc=com"
attribute="uid"
server="ldap://localhost"
allowpattern="Guest*"
killreason="Access denied"
searchscope="subtree"
binddn=""
bindauth=""
verbose="yes"
userfield="yes">
<bind address="" port="9799" type="servers">setup a <link> section to define the server connection. The same thing needs to be setup on the other server to be connected.
<link name="irc.otherdomain.com"Lastly, one of the two servers can be set to <autoconnect> to avoid manually maintaining the connection.
ipaddr="irc.otherdomain.com"
port="9799"
sendpass="secret"
recvpass="secret">
<autoconnect period="60" server="irc.otherdomain.com">
Part 2 of this blog entry will setup the web interface
Wednesday, June 5, 2013
Self Service Password
I decided to allow users on the server to change their passwords, when they want, through a web based tool. I chose LTB's Self Service Password. A simple php tool with lots of neat features like SMS reset, security questions, etc. I only plan to enable the simple form to reset the password. To install, download the latest .deb file, (0.8 in my case). Next install the dependencies, and restart php5.
sudo apt-get install apache2 php5 php5-ldap php5-mcryptThen the .deb
sudo service php5-fpm restart
sudo dpkg -i self-service-password_0.8-1_all.debYou will need to modify php config file at /usr/share/self-service-password/conf/config.inc.php and make some changes to LDAP.
In my case, the server runs on the localhost.
$ldap_url = "ldap://localhost";ldap_binddn and ldap_bindpw are made blank ("") to not use admin credentials.
$ldap_binddn = "";ldap_base is set to your domain.
$ldap_bindpw = "";
$ldap_base = "dc=domain,dc=com";I'm using simple posix schema for users.
$ldap_filter = "(&(objectClass=posixAccount)($ldap_login_attribute={login}))";Next up, modify your nginx config file at /etc/nginx/sites-enabled/default,and add the following sections.
#Self Service Password Section
location /self-service-password {
alias /usr/share/self-service-password;
index index.html index.php;
}
location ~ ^/self-service-password/.*\.php$ {Restart nginx and browse to https://www.domain.com/self-service-password
root /usr/share;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $request_filename;
include /etc/nginx/fastcgi_params;
}
sudo service nginx restart
Tuesday, June 4, 2013
Add Authentication to Nginx
The plan is to expose certain web apps behind Basic HTTP Authentication. This is why the web server is only available via HTTPS, we do not want LDAP user/passwords going over plaintext. The way we have setup LDAP and PAM earlier, it is very easy to secure subdomains using it. Note: Basic HTTP Authentication has uses no session cookies or persistence, so the user remains logged in until they close their browser. Be warned!
Make sure that you have nginx-extras installed, and not nginx. Extras includes the PAM module.
To protect everything under /secure you will add the following to the nginx.conf file: This is secure enough for many purposes.
As an example, to authenticate users against an LDAP server (using the pam_ldap.so module) you will use an /etc/pam.d/nginx like the following:
Make sure that you have nginx-extras installed, and not nginx. Extras includes the PAM module.
sudo apt-get install nginx-extrasThe following is taken from this readme.
To protect everything under /secure you will add the following to the nginx.conf file: This is secure enough for many purposes.
location /secure {Note that the module runs as the web server user, so the PAM modules used must be able to authenticate the users without being root; that means that if you want to use the pam_unix.so module to authenticate users you need to let the web server user to read the /etc/shadow file if that does not scare you (on Debian like systems you can add the www-data user to the shadow group).
auth_pam "Secure Zone";
auth_pam_service_name "nginx";
}
As an example, to authenticate users against an LDAP server (using the pam_ldap.so module) you will use an /etc/pam.d/nginx like the following:
auth required pam_ldap.soIf you also want to limit the users from LDAP that can authenticate you can use the pam_listfile.so module; to limit who can access resources under /restricted add the following to the nginx.conf file:
account required pam_ldap.so
location /restricted {Use the following /etc/pam.d/nginx_restricted file:
auth_pam "Restricted Zone";
auth_pam_service_name "nginx_restricted";
}
auth required pam_listfile.so onerr=fail item=user \And add the users allowed to authenticate to the /etc/nginx/restricted_users (remember that the web server user has to be able to read this file).
sense=allow file=/etc/nginx/restricted_users
auth required pam_ldap.so
account required pam_ldap.so
Build your ownCloud
ownCloud is a way to bring your own cloud storage to the internets. You may want to use this to control your own data, use open standards provided by ownCloud, or just save yourself the monthly subscription costs of the commercial options. The only limitation on storage size is the size of your connected storage, which can actually include other cloud storage services such as Dropbox and Google Drive, in addition to external storage.
Installation is fairly easy with Ubuntu as a repository with packages is available. The following directories are taken from the installation page on ownCloud. Run the following as root
Next step is to link ownCloud to your Ldap server for authentication. Login with the admin account, click the settings button, and go to "Apps". Enable the app for "Ldap User and Group Backend".
Click the settings button, and go to "Admin". Under the LDAP section, set your LDAP host, your domain, and the user and group attributes. Test the configuration and save.
Again, if you are using ownCloud 4.5 everything should work out of the box as is, and users can login and share files with group members. In my case, all web users are a member of the group 'webuser'. However, ownCloud 5.0 requires some additional configuration, or the users are not associated with their groups. The solution is to add the memberUid attribute to the associated group in ldap-account-manager, manually add the users to this group, then tell ownCloud to use this attribute.
Login to ldap-account-manager and click on "Tree View". From here, select the group, and click "Add New Attribute". Select "memberUid". Add the name of at least one user. The new attribute should be visible in the group in tree view. From here, you can manually add members by clicking "Modify Group Members" under memberUid. You can add the users in a batch, instead of manually typing them out.
Log back in to ownCloud as admin. Click on the settings button, and go back into "Admin". In the LDAP section, select the "Advanced" tab. Under "Directory Settings" set "Group-member association" to "memberUid". Save. You may need to remove the users so that ownCloud can repopulate the list with the new group association.
Installation is fairly easy with Ubuntu as a repository with packages is available. The following directories are taken from the installation page on ownCloud. Run the following as root
echo 'deb http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_12.04/ /' >> /etc/apt/sources.list.d/owncloud.listIf you want to add the key to apt-get to avoid a warning
apt-get update
apt-get install owncloud
wget http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_12.04/Release.keyOnce installed, you need to add ownCloud to Nginx. You will presumably already setup php5-fpm during the instructions in my LDAP server post. With ownCloud 5, some slightly more complicated Nginx rules are needed. Again, we are putting ownCloud in a subdomain on the webserver for clean separation of server services. Note: make sure your fastcgi_pass matches the mechanism you are using for FastCGI, either unix socket or tcp socket. The following was taken from this post.
apt-get add - < Release.key
#owncloud settingsMake sure to restart nginx as usual, sudo service nginx restart. You should now be able to create a default admin account at your domain.com/owncloud
#Some rewrite rules, more to come later
rewrite ^/owncloud/caldav((/|$).*)$ /owncloud/remote.php/caldav$1 last;
rewrite ^/owncloud/carddav((/|$).*)$ /owncloud/remote.php/carddav$1 last;
rewrite ^/owncloud/webdav((/|$).*)$ /owncloud/remote.php/webdav$1 last;
location ~ ^/owncloud/(data|config|\.ht|db_structure.xml|README) {
deny all;
}
# Configure the root location with proper rewrite rule
location /owncloud/ {
rewrite ^/owncloud/.well-known/host-meta /public.php?service=host-meta last;
rewrite ^/owncloud/.well-known/host-meta.json /public.php?service=host-meta-json last;
rewrite ^/owncloud/.well-known/carddav /remote.php/carddav/ redirect;
rewrite ^/owncloud/.well-known/caldav /remote.php/caldav/ redirect;
rewrite ^/owncloud/apps/calendar/caldav.php /remote.php/caldav/ last;
rewrite ^/owncloud/apps/contacts/carddav.php /remote.php/carddav/ last;
rewrite ^/owncloud/apps/([^/]*)/(.*\.(css|php))$ /index.php?app=$1&getfile=$2 last;
rewrite ^(/owncloud/core/doc[^\/]+/)$ $1/index.html;
try_files $uri $uri/ index.php;
}
# Configure PHP-FPM stuff
location ~ ^(?<script_name>.+?\.php)(?<path_info>/.*)?$ {
try_files $script_name = 404;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_param PATH_INFO $path_info;
fastcgi_param HTTPS on;
# This one is a little bit tricky, you need to pass all parameters in a single line, separating them with newline (\n)
fastcgi_param PHP_VALUE "upload_max_filesize = 1024M \n post_max_size = 1024M"; # This finishes the max upload size settings
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; # On some systems OC will work without this setting, but it doesn't hurt to leave it here
include /etc/nginx/fastcgi_params;
}
Next step is to link ownCloud to your Ldap server for authentication. Login with the admin account, click the settings button, and go to "Apps". Enable the app for "Ldap User and Group Backend".
Click the settings button, and go to "Admin". Under the LDAP section, set your LDAP host, your domain, and the user and group attributes. Test the configuration and save.
Again, if you are using ownCloud 4.5 everything should work out of the box as is, and users can login and share files with group members. In my case, all web users are a member of the group 'webuser'. However, ownCloud 5.0 requires some additional configuration, or the users are not associated with their groups. The solution is to add the memberUid attribute to the associated group in ldap-account-manager, manually add the users to this group, then tell ownCloud to use this attribute.
Login to ldap-account-manager and click on "Tree View". From here, select the group, and click "Add New Attribute". Select "memberUid". Add the name of at least one user. The new attribute should be visible in the group in tree view. From here, you can manually add members by clicking "Modify Group Members" under memberUid. You can add the users in a batch, instead of manually typing them out.
Log back in to ownCloud as admin. Click on the settings button, and go back into "Admin". In the LDAP section, select the "Advanced" tab. Under "Directory Settings" set "Group-member association" to "memberUid". Save. You may need to remove the users so that ownCloud can repopulate the list with the new group association.
Sunday, March 3, 2013
Setting up LDAP Server
LDAP Server Setup
Lightweight Directory Access Protocol or LDAP, is a high-level application protocol for managing directory services in a hierarchical manner. It's most common use is to manage domain related information, such as an email directory or user information. In my case, I will use the Unix-related structures for managing users and their system access to my services. The same user name and login will be used for Subsonic, Owncloud, COPS (e-book server), SSH-SFTP logins, etc.
To start installing OpenLDAPServer, you can use 2 guides over at Ubuntu, here and here. The second link actual corrects a few things in the guide, but most of it is unnecessary for an initial LDAP server.
First off, install the packages.
User Configuration
A great utility for managing users and settings includes a web app called LAM, or ldap-account-manager. There is another great write-up regarding using LAM with Nginx here.
Install the necessary packages
Lightweight Directory Access Protocol or LDAP, is a high-level application protocol for managing directory services in a hierarchical manner. It's most common use is to manage domain related information, such as an email directory or user information. In my case, I will use the Unix-related structures for managing users and their system access to my services. The same user name and login will be used for Subsonic, Owncloud, COPS (e-book server), SSH-SFTP logins, etc.
To start installing OpenLDAPServer, you can use 2 guides over at Ubuntu, here and here. The second link actual corrects a few things in the guide, but most of it is unnecessary for an initial LDAP server.
First off, install the packages.
sudo apt-get install slapd ldap-utilsIn my case, my host is already joined to a domain, so I didn't need the next step, but just to make sure, reconfigure slapd to add the ldap domain and reset the password.
sudo dpkg-reconfigure slapdThat's pretty much all you need to get running. The latest builds of Ubuntu handle the inclusion of various basic schemas, but to verify your ldap is up and running, run the following.
sudo ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=configInitially this command should list 10-15 entries and is a good first check.
User Configuration
A great utility for managing users and settings includes a web app called LAM, or ldap-account-manager. There is another great write-up regarding using LAM with Nginx here.
Install the necessary packages
sudo apt-get install php5-fpm php5 php5-ldap php-apc php5-gd php-fpdf ldap-account-managerNormally php5-fpm is configured listening on 127.0.0.1 port 9000. We're going to change this to a Unix socket, just to clean up the ports a bit and potentially increases performance under load. In general it won't help much, but theoretically removes some of the TCP overhead.
sudo vi /etc/php5/fpm/pool.d/www.confLook for
listen = 127.0.0.1:9000Change to
listen = /var/run/php5-fpm.sockRestart the service
sudo service php5-fpm restartAdd the following section to /etc/nginx/sites-enabled/default to create a sub-domain for the account manager, and will point it to the main launch page.
location /ldap-account-manager {Add the following section to /etc/nginx/sites-enabled/default to point Nginx to the LAM directory, the php Unix socket, and tweak a couple of fastcgi parameters.
alias /usr/share/ldap-account-manager;
index index.html index.php;
}
location ~ ^/ldap-account-manager/.*\.php$ {Restart Nginx
root /usr/share;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $request_filename;
include /etc/nginx/fastcgi_params;
}
sudo service nginx restart
You should be able to now browse to LAM https://www.domain.com/ldap-account-manager. At this point, I refer you to the ducky-pond.com post for how to initially setup LAM.
Client Access
First, configure your client config for ldap client apps.
First install the packages
Client Access
First, configure your client config for ldap client apps.
sudo vi /etc/ldap.confMake sure the domain is properly specified
base dc=domain,dc=comand the uri is correct
uri ldap://127.0.0.1:389I refer you to this thread for the instructions I used to setup local and ssh logins for your users. This will automatically create their home directories if they do not exist. There is a correction by a later contributer, which I have included in my quick setup below
First install the packages
sudo apt-get install ldap-utils libpam-ldap libnss-ldap nslcdNext edit /etc/nsswitch.conf and change the lines for passwd, group, and shadow
passwd: compat ldapEdit /etc/pam.d/lightdm and add
group : compat ldap
shadow: compat ldap
session required pam_mkhomedir.so skel=/etc/skel umask=0022Edit /etc/pam.d/common-session and add
session required pam_mkhomedir.so skel=/etc/skel umask=0022Apply changes
sudo update-rc.d nslcd enableConfigure lightdm to allow user to specify a username for login
sudo /usr/lib/lightdm/lightdm-set-defaults -m trueAnd reboot. If the user logs in locally or via ssh, her home directory will be created automatically.
Subscribe to:
Posts (Atom)