Jimmy B. - How To Install Dreamwidth

Ever since 2007 or so I've been running and managing Scribbld.com. Since then we've been using the LiveJournal open source code to run the site. In the beginning the code was managed by a group of really great people and developers, however during the past 10 years or so LiveJournal has swapped hands far more then I can count. Even in the beginning the code wasn't all that easy to manage or upgrade, so much so that I believe I only updated the code-base once. It was nothing short of a nightmare.

After much thought and deliberation I've decided to switch my LiveJournal-based site over to using the Dreamwidth-based code. Dreamwidth utilized the LiveJournal code a few years back and has forked it and done some really awesome stuff with it. Most (if not all) of it remains to be BML/Perl which I really dislike but at least it's continually being updated and is managed at GitHub which I really enjoy using.

For those wishing to run Dreamwidth (or rather a Dreamwidth clone if you may) this guide should help a lot of you to get going. This guide is heavily based on the one Dreamwidth has provided however it includes a few new things.

Firstly you'll need your own server with which you have root access too. I used Ubuntu 14.04 x86_64 without any issues. It has about 160GBs of disk space and 4GBs of RAM. Remember, Perl likes to eat all your memory, so having lots of RAM is going to help a lot.

Update Your System

I started off with a fresh Ubuntu 14.04 install, a clean, blank slate. However, it's also important to make sure your system software is up to date. Simple.

apt-get update
apt-get upgrade

If you'd like to have man pages on your system (may already be installed):

apt-get install man

Next you'll need a bunch more software to run Dreamwidth.

Installing Necessary Packages

apt-get install git-core subversion apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim libclass-autouse-perl \
  libdatetime-perl libcache-memcached-perl libhash-multivalue-perl \
  libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \
  libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \
  libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \
  libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \
  libgnupg-interface-perl libmail-gnupg-perl perlmagick \
  libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \
  libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \
  libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \
  libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \
  libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \
  libtemplate-perl libterm-readkey-perl \
  gcc libtest-most-perl libgearman-client-perl \
  libbusiness-creditcard-perl liblwpx-paranoidagent-perl \
  libtheschwartz-perl libfile-type-perl libjson-perl

Their instructions also included libtest-simpleunit-perl but I found this package was not necessary and haven't come across any issues without having installed it. This will download about 110MBs of data and then install it to your system. During this time you may be asked to setup MySQL, however it's very simple, and should only ask a couple questions such as setting up the root password for MySQL.

Set Apache to Use MPM Pre-fork

If you've used the above one-liner it should install the Prefork module for Apache, however it may not configure it to use it. By default my copy of Apache was set to use MPM Event. Easy enough to change though.

a2dismod mpm_event
a2enmod mpm_prefork

Install Extra Perl Modules

Next you'll need to install a few more Perl modules the Dreamwidth code will need to run. Each line below is a single command. To save you from having to run the first one twice, you should run these commands first:

locale-gen en_US en_US.UTF-8
dpkg-reconfigure locales

This just configures language locales on the system, which the Perl modules appear to require before being installed.

cpan Bundle::CPAN   # this will guide you through setting up CPAN
cpan GTop  
cpan Digest::SHA1
cpan Unicode::CheckUTF8
cpan MogileFS::Client   # this is necessary even if you don't use MogileFS
cpan TheSchwartz::Worker::SendEmail
cpan TheSchwartz::Worker::PubSubHubbubPublish
cpan Digest::SHA256
cpan Text::Markdown
cpan Image::ExifTool

Install Postfix

Next install Postfix and configure it. The configuration is relatively easy to go through. I won't get into how to configure it, there's Google for that.

apt-get install postfix

Set Timezone to UTC

For whatever reason Dreamwidth requires the timezone to UTC.

dpkg-reconfigure tzdata

Setup a 'dw' User Account

Add the account to the system:

adduser dw

Next add the user to the sudo group:

usermod -a -G sudo dw

Get the Dreamwidth Code!

These next few commands will have you run them as the dw user account.

su - dw
export LJHOME=/home/dw/dw
echo $LJHOME

The final command above should return /home/dw/dw.

Next you'll need to get yourself a free GitHub account. Once that is done fork the follow repositories:

Here's a quick overview of how the repositories will work together:

dw-free is the main repository and goes into $LJHOME
other repositories will go into $LJHOME/ext
personal config files go into $LJHOME/ext/local
All code from dw-free ($LJHOME) and the additional repos under $LJHOME/ext are automatically live when you start your server. There's no need to run any additional syncing steps anymore.

Setup dw-free

# clone a copy of the repository onto your machine
cd ~/
git clone https://USERNAME@github.com/USERNAME/dw-free.git $LJHOME
cd $LJHOME

# and let's make it aware of the dreamwidth repository so we can grab updates later
git remote add dreamwidth https://github.com/dreamwidth/dw-free
git fetch dreamwidth
# we also want to sync up with the main branches
git branch --set-upstream develop dreamwidth/develop 
git branch --set-upstream master dreamwidth/master

You'll want to replace the USERNAME bits with your GitHub username. This will make your repository an authenticated version that can push changes back to Github, as opposed to only being able to pull them.

I am leaving out the setup the dw-nonfree code since I did not use that repository, however you can refer to their guide for more information on how to set it up.

Database Setup

MySQL should already be installed, setup and running at this point. Next you'll need to create a couple databases. First get logged into MySQL.

mysql -u root -p

Next you'll run a couple commands to actually setup the database(s):

create database dw;
grant all on dw.* to 'dw'@'localhost' identified by 'SomePassword';

Next you'll add a database for TheSchwartz.

create database dw_schwartz;
grant all on dw_schwartz.* to 'dw'@'localhost' identified by 'SomePassword';

You can then quit out of MySQL.

Setting Up The Configuration Files

The next part may be the most time consuming part. You'll need to edit your configuration files. There are three really important configuration files for the whole install, so it's good to learn which configuration values are store in each file.

## copy over your config files, and tell them to take priority
## over config files in any of the repos
cd $LJHOME
mkdir -p ext/local/etc
echo "highest"  > ext/local/.dir_scope
cp doc/config-local.pl.txt ext/local/etc/config-local.pl
nano ext/local/etc/config-local.pl

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. You will want to change at least your $DOMAIN and the %DBINFO structure.

cd $LJHOME
cp doc/config-private.pl.txt ext/local/etc/config-private.pl
chmod go-rwx ext/local/etc/config-private.pl
nano ext/local/etc/config-private.pl

Note that there are two DBINFO blocks; make sure to get them both.

Run checkconfig.pl

Once you've updated your configurations you'll want to run a check script which looks over your install to make sure any necessary packages and software have been installed. I believe the first time I ran this one or two packages came up missing. Fortunately the script let's you know which packages are missing and even provides necessary commands to get them installed.

cd $LJHOME
bin/checkconfig.pl --no=ljconfig

Populate the database

Now let's get some structure and initial data into the database.

$LJHOME/bin/upgrading/update-db.pl -r --innodb
$LJHOME/bin/upgrading/update-db.pl -r --innodb # at least for now we have to run this twice
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/update-db.pl -p

Next, you'll populate the dw_schwartz database:

mysql -u dw -p dw_schwartz

Create a system user account

Now, we'll create a initial system account. This lets you login as the system user and generate invite codes and set privileges to other user accounts.

$LJHOME/bin/upgrading/make_system.pl

Load Translation Strings

Next, you'll run a tool to load in translations strings.

$LJHOME/bin/upgrading/texttool.pl load

Configure Apache

You will need configure Apache next. You'll want to do this as the root user account. Put the following into /etc/apache2/sites-available/stage.conf:

User dw
Group dw
UseCanonicalName off

StartServers 1
MaxSpareServers 2
MinSpareServers 1

DocumentRoot /home/dw/dw/htdocs

PerlSetEnv  LJHOME /home/dw/dw
PerlPassEnv LJHOME

PerlRequire  /home/dw/dw/cgi-bin/modperl.pl

Now you'll want to disable the default site, and enable this new site:

a2dissite default
a2ensite stage

Go ahead and restart Apache. Hopefully you'll get no errors.

/etc/init.d/apache2 restart

Extra Notes

It appears for some reason the default language is set to en_DW in the $LJHOME/etc/config.pl. Change:

@LANGS = qw( en_DW );

@LANGS = qw( en );

en_DW to en

There also appears to be an issue with some of the code utilizing remote_ip instead of client_ip. It's possible this is due to the code being older and being developed for older versions of Apache then what I have installed. Fortunately it's easy to fix. You'll need to edit a bit of code here. On line 173 of /home/dw/dw/cgi-bin/DW/Request/Apache2.pm:

# returns the ip address of the connected person
sub get_remote_ip {
    my DW::Request::Apache2 $self = $_[0];
    return $self->{r}->connection->remote_ip;
}

# returns the ip address of the connected person
sub get_remote_ip {
    my DW::Request::Apache2 $self = $_[0];
    return $self->{r}->connection->client_ip;
}

Next in /home/dw/dw/cgi-bin/Apache/BML.pm on line 1919 change:

sub get_remote_ip
{
    my $apache_r = BML::get_request();
    return $apache_r->connection()->remote_ip;
}

sub get_remote_ip
{
    my $apache_r = BML::get_request();
    return $apache_r->connection()->client_ip;
}

remote_ip -> client_ip

Getting the Shopping Cart Working

If you'd like to get the basic shopping cart working, you'll first want to setup API keys for PayPal. Just login to your PayPal account, go to Tools and then under Manage your business click on API Access. Go through the setup process there. Once finished you'll end up with an API username, password and signature. Note, that some menu options or page locations may be different within PayPal when you read this. You'll copy those into the $LJHOME/ext/local/etc/config-private.pl file. Now open up the $LJHOME/ext/local/etc/config-local.pl file and locate %SHOP. You can edit this to your liking (setting pricing and such). It should look something like this when you're done:

# shop/pricing configuration
    %SHOP = (
        # key => [ $USD, months, account type, cost in points ],
        prem6  => [  20,  6, 'premium', 200 ],
        prem12 => [  40, 12, 'premium', 400 ],
        paid1  => [   3,  1, 'paid', 30    ],
        paid2  => [   5,  2, 'paid', 50    ],
        paid6  => [  13,  6, 'paid', 130   ],
        paid12 => [  25, 12, 'paid', 250   ],
        seed   => [ 200, 99, 'seed', 2000   ],
        points => [],     # if present, sell points
        vgifts => [],     # if present, sell virtual gifts
        rename => [ 15, undef, undef, 150 ],
    );

Getting YouTube API

If you'd like to utilize YouTube/embeds on your site, first edit the $LJHOME/ext/local/etc/config-local.pl and edit this:

# additional domain from which to serve the iframes for embedded content
# for security reasons, we strongly recommend that this not be on your $DOMAIN
$EMBED_MODULE_DOMAIN = "embed.your-domain.com";

I have two domains, so I just put the URL on the secondary domain.

Next, you'll need to get an API access key to YouTube. For this I ended up Googling 'YouTube API key'. Basically you just need to go here and register your site and choose the YouTube Data API. Copy over your key(s) into the $LJHOME/ext/local/etc/config-private.pl file and you should be all set.

Adding Privs

As a rule of thumb I don't use the system account. I setup my own account and add any necessary privileges onto the account which I may need. The ones I used are as follows. To add these to my personal account, I do this from the system account $URL/admin/priv/. Here's what I add. They're mostly privileges which let me manage the FAQs and help requests.

faqadd *
faqcat
faqedit *
payments
siteadmin
supportchangesummary
supportclose
supporthelp
supportmakeinternal
supportmovetouch
supportread
supportviewinternal
supportviewscreened
supportviewstocks
suspend
sysban
vgifts *

Setting Up Workers

For your website to be fully functional you'll want to configure some workers to handle various jobs around your website. I mostly just went off the wiki page for this.

How To Install Dreamwidth