Installing the latest NodeJS & MongoDB Combo

NodeJS

Requirements

These are needed for building and running.

sudo apt-get install build-essential python

Install

https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager.

OR

Build & Install

download the latest NodeJS source code.

wget http://nodejs.org/dist/v0.10.21/node-v0.10.21.tar.gz

Decompress and enter the directory.

tar -xvzf node-v0.10.21.tar.gz
cd node-v0.10.21

Configure the make file for my machine then build and install as a root user.

./configure
make
sudo make install

Test to make sure it was installed.

node -v
npm version

Optional: install MongoJS as a module that implements the offical Mongo API.

npm install mongojs

MongoDB

I should follow the instructions on the official MongoDB manual.

What things do I need for a RESTful hypermedia architecture (RMM Level 3)

For my work I need to find out what is necessary for a modern REST-API. In theory, I want to get a global view about the glory of REST “Richardson Maturity Model“.

First, I want to thank Matthew Weier O’Phinney for the documentation about his PhlyRestfully module for the Zend Framework 2. PhlyRestfully implements RESTful JSON APIs in Hypermedia Application Language (HAL). He also explain things under the hood.

It should be forgiven, that I will use many words from his documentation for a overview in this lazy article. :-)

HAL Primer
HAL, short for “Hypermedia Application Language”, is an open specification describing a generic structure for RESTful resources.
- expose resources
- via HTTP, using HTTP verbs to manipulate them
- and provide canonical links to themselves, as well as link to other, related resources

Hypermedia Type
HAL presents two hypermedia types, one for XML and one for JSON. Typically, the type is only relevant for resources returned by the API, as relational links are not usually submitted when creating, updating, or deleting resources. The generic mediatype that HAL defines for JSON APIs is “application/hal+json”.

Resources
For JSON resources, the minimum I must do is provide a “_links” property containing a “self” relational link. As an example:

{
    "_links": {
        "self": {
            "href": "http://example.org/api/users/1"
        }
    }
    "id": "1",
    "name": "Homer Simpson"
}

Interacting with HAL
a request, using the Accept header with a value of “application/json” or “application/hal+json” (the latter really isn’t necessary, though).
If POST’ing, PUT’ting, PATCH’ing, or ‘DELETE’ing a resource, you will usually use a Content-Type header of either “application/json”, or some vendor-specific mediatype you define for your API; this mediatype would be used to describe the particular structure of your resources without any HAL “_links”. Any “_embedded” resources will typically be described as properties of the resource, and point to the mediatype relevant to the embedded resource. The API will respond with a mediatype of “application/hal+json”.

API-Problem
How do I reporting errors?
REST advocates indicate that HTTP response status codes should be used, but little has been done to standardize on the response format.
This mediatype, “application/api-problem+json” is via the IETF, and actually also includes an XML variant.
- problemType: a URL to a document describing the error condition (required)
- title: a brief title for the error condition (required)
- httpStatus: the HTTP status code for the current request (optional)
- detail: error details specific to this request (optional)
- problemInstance: a URL to the specific problem occurrence (e.g., to a log message) (optional)

HTTP/1.1 500 Internal Error
Content-Type: application/api-problem+json
 
{
    "problemType": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html",
    "title": "Internal Server Error",
    "detail": "Status failed validation",
    "httpStatus": 500
}

Whitelisting HTTP Methods
should I provide with the via HTTP method, OPTIONS, and a related HTTP response header, Allow. Calls to OPTIONS are non-cacheable, and may provide a response body if desired. They should emit an Allow header, however, detailing which HTTP request methods are allowed on the current URI.

OPTIONS /api/users
Host: example.org

with it’s response:

HTTP/1.1 200 OK
Allow: GET, POST

Content Negotiation
Should a client inform the server what media types it understands. More precisely, the client provides an Accept HTTP header that lists acceptable media types and associated quality factors. It can choose the best representation of a resource. see: http://en.wikipedia.org/wiki/Content_negotiation

the accept definition in our request should be: “*/json”

Service Versioning via Content Negotiation
Inevitably there will come a time when an API requires a change to its returned or expected representation that will cause consumers to break and that breaking change must be avoided. Versioning the API is the way to avoid breaking clients and consumers.

# Request
GET http://example.com/api/users/12345
Accept: application/json; version=1
 
# Response
HTTP/1.1 200 OK
Content-Type: application/json; version=1
{"id":"12345", "name":"Homer Simpson"}

Metadata Mapping
Do I need a low-level, configurable mechanism that ensure:
- resources have the correct “self” relational link
- resources are extracted to a JSON representation correctly
- that embedded resources are rendered as embedded HAL resources

The metadata map maps a class to a set of “rules” that define whether the class represents a resource or collection, the information necessary to generate a “self” relational link etc.

CRUD, Verbs, and Actions

GET /users              index   display a list of all users
GET /users/new          new     return an HTML form for creating a new user
POST /users             create  create a new user
GET /users/:id          show    display a specific user
GET /users/:id/edit     edit    return an HTML form for editing a user
PATCH/PUT /users/:id    update  update a specific user
DELETE /users/:id       destroy delete a specific user

resource names are in plural

OPTIONAL: ETags
There two uses for ETags (Entity Tags):
- Conditional GET (the so-called “cacheing mechanism”)
- Concurrency Control (resolves “Lost Update Problem”)

Libraries
HAL
API-Problem
Backbone.HAL
ZF2.PhlyRestfully

Connecting Drupal to Solr Server

Happily, Solr also plays nicely with Drupal. So my colleagues want to connect their Drupal application to the installed Solr server (see my last blog entry). Drupal provide two modules for Solr server integration. The modules links external Solr server with the Drupal application, passing data into Solr to index, and then enabling Drupal to serve up the search results. As of the writing of this how-to two modules have an advantage and a disadvantage. The first is Search API Solr search with the advantage you can use the Solr Index straight in views. The disadvantage is you can not index files remotely on the Solr server https://drupal.org/node/1289222. The second module is Apache Solr search integration with the advantage you can index file attachments remotely on the Solr server. The disadvantage is you can not use the search index for views.

Okay however which module we choose, the installation of the configuration files are the same.

1
2
wget http://ftp.drupal.org/files/projects/apachesolr-7.x-1.3.tar.gz
tar -zxvf apachesolr-7.x-1.3.tar.gz

Next I have to copy the module configuration files from /solr-conf/solr-4.x/* to /opt/apache-solr-4.3.1/my-app/solr/collection1/conf/. So I restart the Solr server to check if there any log messages appears. In my case, currently the solrconfig.xml do not know the “collection1″ directory, so the relative path of the “../../contrib” directory is wrong and must be set one level up to “../../../contrib” in the solrconfig.xml.

Next my colleagues install the module in Drupal and setting up the Solr server URL: http://X.X.X.X:8983/solr/collection1

Apache Solr Attachments

The Apache Solr attachments module lets you extract documents using Solr server. For that I need to configure Solr accordingly and edit the solrconfig.xml to have these lines:

<lib dir="../../../contrib/extraction/lib" />
<lib dir="../../../dist/" regex="solr-cell-\d.*\.jar" />
 
<!-- An extract-only path for accessing the tika utility -->
<requestHandler name="/extract/tika" class="org.apache.solr.handler.extraction.ExtractingRequestHandler" startup="lazy">
  <lst name="defaults">
  </lst>
  <!-- This path only extracts - never updates -->
  <lst name="invariants">
    <bool name="extractOnly">true</bool>
  </lst>
</requestHandler>

https://drupal.org/node/1205190
Note: For locally extracting documents I need the Apache Tika app on the Drupal webserver.

How to set up Apache Solr 4.X

Solr is an open source search server based on Apache Lucene. For my work at Inostudio I have to install Solr on a clean server. The Solr platform is written in Java and I need to start by installing Java:

apt-get install openjdk-7-jre

Solr can also be run in a Tomcat environment but I do prefer do not embed Solr in a Tomcat container. Maybe it make sense if I have already a Tomcat running on the server.

Install Apache Solr

Apache Solr can be download from http://ftp.download-by.net/apache/lucene/solr/ and as of this writing it’s version 4.3.1.

1
2
3
4
wget http://ftp.download-by.net/apache/lucene/solr/4.3.1/solr-4.3.1.tgz
tar -zxvf solr-4.3.1.tgz
cp -rf solr-4.3.1 /opt/apache-solr-4.3.1
mv /opt/apache-solr-4.3.1/example /opt/apache-solr-4.3.1/my-app

Now I can set up the solr configs xml in the my-app/solr/collection1 directory. For the first time I should be able to connect to Solr by starting it from the command line.

1
2
3
cd /opt/apache-solr-4.3.1/my-app
java -jar start.jar
http://X.X.X.X:8983/solr

With “Ctrl+c” I can shutdown Solr after I have check the connection.

Run Solr automatically

At this point I running Solr manually, which I probably do not want to do on a production site. To get it to start on startup, I need a startup script. But before I write the script I need the daemon tool.

apt-get install daemon
nano -w /etc/init.d/solr
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
#!/bin/sh
### BEGIN INIT INFO
# Provides:          solr
# Required-Start:    $local_fs $remote_fs $network $syslog $named
# Required-Stop:     $local_fs $remote_fs $network $syslog $named
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# X-Interactive:     true
# Short-Description: Start/Stop Apache Solr
### END INIT INFO
 
# This script will launch Solr in a mode that will automatically respawn if it
# crashes. Output will be sent to $LOG. A pid file will be
# created in the standard location.
 
NAME=solr
SOLR_DIR=/opt/apache-solr-4.3.1/my-app
COMMAND="java -jar start.jar"
LOG=/var/log/solr.log
 
start () {
    echo -n "Starting solr..."
 
    # start daemon
    daemon --chdir=$SOLR_DIR --command "$COMMAND" --respawn --output=$LOG --name=$NAME --verbose
 
    RETVAL=$?
    if [ $RETVAL = 0 ]
    then
        echo "done."
    else
        echo "failed. See error code for more information."
    fi
    return $RETVAL
}
 
stop () {
    # stop daemon
    echo -n "Stopping $NAME..."
 
    daemon --stop --name=$NAME  --verbose
    RETVAL=$?
 
    if [ $RETVAL = 0 ]
    then
        echo "done."
    else
        echo "failed. See error code for more information."
    fi
    return $RETVAL
}
 
restart () {
    daemon --restart --name=$NAME  --verbose
}
 
status () {
    # report on the status of the daemon
    daemon --running --verbose --name=$NAME
    return $?
}
 
case "$1" in
    start)
        start
    ;;
    status)
        status
    ;;
    stop)
        stop
    ;;
    restart)
        restart
    ;;
    *)
        echo $"Usage: $NAME {start|status|stop|restart}"
        exit 3
    ;; esac
 
exit $RETVAL

After saving the script above I have to change its permission so it can be executed and add it to the different run-levels.

1
2
3
chmod 755 /etc/init.d/solr
/etc/init.d/solr start
update-rc.d solr defaults

Multicore

Since Inostudio have more sites running, hence I configure Solr to run multicore so I can use the same Solr to index more sites on the external server.
After copy the /opt/apache-solr-4.3.1/my-app/solr/collection1 directory to /opt/apache-solr-4.3.1/my-app/solr/collection2 directory I edit the /opt/apache-solr-4.3.1/my-app/solr/solr.xml to add a new core line:

<core name="collection2" instanceDir="collection2" />

Additional

1. For Solr security I should protect the server by basic http authentication.
2. To avoid the warning “Can’t find (or read) directory to add to classloader:
/non/existent/dir/yields/warning” I have to remove this entry in the solrconfig.xml

<lib dir="/non/existent/dir/yields/warning" />

Links

Zend_Session Error Message showing up randomly

I’m currently working on a new application using still Zend-Framework 1 but, for whatever reason, this Error Message is showing up at any location totally randomly.

Fatal error: Uncaught exception 'Zend_Session_Exception' with message 'Zend_Session::start()
- /var/www/Foobar/src/library/Zend/Session.php(Line:469): Error #8 session_start() [<a href="function.session-start">function.session-start</a>]: ps_files_cleanup_dir: opendir(/tmp) failed: Permission denied (13) Array' in etc.

A solution is to set the session.save_path in the php.ini file to a writable directory. For our hoster, the save_path was set to /tmp and whatever reason, the session_garbage_collector (session.gc_probability is enabled) could not clean up the session directory.
The php.ini setting session.gc_probability in conjunction with session.gc_divisor is used to manage probability that the gc (garbage collection) routine is started.

See also manual: http://php.net/manual/en/session.configuration.php

International PHP Conference 2013 Day 2

Day 2 of IPC 2013 was just exciting as the first day so I have attended following talks: ipc13

keynote #3 – The end of the control in the digital age (webinale)
Prof. Bernhard Pörksen, University Tübingen

- Radikale Demokratisierung von Enthüllungs- und Empörungspraktiken
- Reputationsbeschädigungen im guten oder schlechten Sinne leicht
Veränderungen:
- neue Akteure (nicht nur mehr Journalisten) blogger köhlerskandal
- neue Themen (Wurstskandal ing-diba)
- neue Opfer (shitdog girl)
- neue Öffentlichkeiten (hilfiger winfrey)
- neue Tools (Allzweckwaffen der Skandalierung) smartphone = indiskrete

Introduction to NodeJS
Sebastian Springer, Mayflower GmbH

http://de.slideshare.net/sspringer82/node-jsdmuc

Jenkins Automated Deployment to Production
Sebastian Bergmann, Arne Blankert, thePHP.cc

http://thephp.cc/termine/2013/international-php-conference-spring-edition/automated-deployment-from-jenkins-to-production

Appointing a Proxy
Stefan Priebsch, thePHP.cc

http://thephp.cc/termine/2013/international-php-conference-spring-edition/appointing-a-proxy

Scaling PHP in the real World
Dustin Whittle, AppDynamics

https://speakerdeck.com/dustinwhittle/scaling-php-in-the-real-world