Useful Tips for Programming: 2014

Thursday, 3 July 2014

Viewing Processes on Linux

Here is a file I intend to update with how to view things that are going on:

ps -ef
Lists processes running with username. Useful followed be a grep to find info about certain users / processes.

top
More interactive list updated in real-time, like a command line version of the MS Window'sTask Manager

kill 4891
kill a process by typing this command followed by its process id from the list.

Friday, 27 June 2014

Adding a host wtih Tomcat verses adding a webapp.

This gets round the fact that the links in VfB are all root, so when running it on localhost the folder is under localhost:8080/vfb/.... so links all miss out the /vfb/ bit. This will change the running location to hachi.ko:8080/ to avoid this problem.

Add a name pointing back to the localhost in the computer's host file /etc/hosts (my computer is called hachiko, hence the "domain" name hachi.ko:
127.0.0.1 hachi.ko

Add to the server.xml file in tomcat's conf directory (/var/lib/tomcat7/conf) in the <service> tag:
<Host name="hachi.ko" appBase="webapps" unpackWARs="true" autoDeploy="true">
<Context path="" docBase="/home/s1144899/vfb/VFB" debug="0" privileged="true" />
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="hachi_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
</Host>

Note how the docBase is the path to the actual folder for the webapp.

There is a folder in the /var/lib/tomcat7/conf/Catalina/hachi.ko directory created called hachi.ko, but we don't need a webapp link script as the path is specified above.

The script we used for that before was called vfb.xml and stored in the localhost folder in /var/lib/tomcat7/conf/Catalina/:
<?xml version="1.0" encoding="UTF-8"?>
<Context path="/vfb"
docBase="/home/s1144899/vfb/VFB"
antiResourceLocking="false" privileged="true" />
This allowed us to link to a folder as opposed to keeping our webapp in the webapp directory specified by tomcat.

Thursday, 26 June 2014

Setting permissions in Linux

Set the permissions of folder in path to be read/write for usergroup groupname for this folder and current and future containing files:

sudo setfacl -Rdm g:groupname:rwx /path/folder

Alternatively set ownership using chown:

sudo chown username filename

Now check privilages using ls -l

Wednesday, 25 June 2014

Compiling Servlets (for Tomcat)

Just a quick one for now...

Install the Javac compiler if you need to,

You need to include in the classpath the Tomcat servlet-api.jar:

Put the servlet in the Tomcat project's src/ directory and then compile using:

javac -classpath /usr/share/tomcat7/lib/servlet-api.jar ServletDemo.java

Now place the class file created in the WEB-INF/classes directory.

Remember to restart Tomcat if you update files:
sudo service tomcat7 restart

An example servlet might be:
package com.fewtrem;

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class ServletDemo extends HttpServlet{
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws IOException{
PrintWriter out = response.getWriter();
out.println("<html><body>Fewtrem's first Servlet</body></html>");
}

}

Understanding Web.xml Files

Tomcat uses a web.xml file in the WEB-INF directory of an application to configure the website.

This was inspired by reading material on http://wiki.metawerx.net/wiki/web.xml and analysing the VfB web.xml file.

The order of tags should be respected.

The minimal tags are:

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
</web-app>

we can then put the following blocks of XML tags into the <web-app> tag:

Web-App Name
<display-name>My First WebApp</display-name>
Name of the web-app - appears in the manager with this name.

Description
<description>The first web-app I created</description>
Description of what the web-app does

Clusters
<distributable />
Add if the app can be used on computer clusters

Session Time-Out
<session-config>
<session-timeout>120</session-timeout>
</session-config>
Set session info, such as time-out (here 120 minutes).

JSP (Java Server Pages) Configuration
<jsp-config>
<jsp-property-group>
<url-pattern>*.jsp</url-pattern>
<include-prelude>/WEB-INF/jspf/prelude1.jspf</include-prelude>
<include-coda>/WEB-INF/jspf/coda1.jspf</include-coda>
</jsp-property-group>
</jsp-config>
Configure the Java Server Pages: Here a header and footer jsp script is added to each .jsp file by using the jsp-property-group tag, specifying the url-pattern as all pages ending in .jsp and using the include-prelude and include-coda tags respectively.

Context (Environment variables)
You can specify variables in web.xml in order to refer to them in the rest of the application, a bit like environment variables. e.g.
<context-param>
<param-name>myImportantEnvironmentInteger</param-name>
<param-value>100</param-value>
</context-param>

Servlet
A Java servlet is a java program that usually serves up HTML when requested by an external user, thus we need to tell the web-app how the user calls it. c.f. a JSP file that is similar but an HTML file with (usually small amounts of) java in it in tags - a bit like PHP.
<servlet>
<servlet-name>springmvc</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<description>This is a servlet that implements the Spring Framework</description>
<load-on-startup>1</load-on-startup>
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value> /WEB-INF/classes/springmvc-servlet.xml </param-value>
</init-param>
</servlet>
We specify the servlets (usually just the ones directly callable by users entering web addresses). They require a name for use in this web-xml (servlet-name) and the address of the class file where they are kept (servlet-class). The init() method will be called when the application starts as the load-on-startup tag is specified. The number determines the order that the servlet is loaded in on start-up. The parameters required for loading can be passed to it by using an init-param tag for each parameter.

Servlet Calling (Mapping)
Add this code after a servlet has been defined
<servlet-mapping>
<servlet-name>springmvc</servlet-name>
<url-pattern>*.html</url-pattern>
</servlet-mapping>
Simply process all requests for HTML files through the springmvc servlet (an external framework made in java servlet form). In general servlet-mapping maps external user specified web-addresses to java servlet programs.

Listeners
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
Listeners are run when certain actions occur in the application e.g. session expiry etc. They need to be specified here as they need to run when the server is running not when called by a user.

Filters
<filter>
<filter-name>charsetFilter</filter-name>
<filter-class>uk.ac.ed.vfb.servlets.CharsetEncodingFilter</filter-class>
</filter>
Filters are extra java programs that are run when the user specifies certain pages, and are generally external to the application.

Filter Mapping
<filter-mapping>
<filter-name>charsetFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
For them to be called by users in addresses we need to map them as with servlets. E.g. the external program charsetFilter is run on all pages.

Error Handling
<error-page>
<error-code>404</error-code>
<location>/error.htm</location>
</error-page>
Map errors based on the server error code to a certain page specified in the location tag.

Thursday, 19 June 2014

Basic Shell Things

These are related to deployment scripts in Virtual Fly Brain.

Find files

Use the simple find command:

find *.txt

finds all files ending in .txt in current directory (note that the * is the wildcard for any bumber of any characters (including none at all).

find -name '*.txt'

finds all files ending in .txt in this and below directories

find data/ -name '*.txt'

finds all files ending in .txt in directory data

find -name '*.txt' -or -name '*.htm'

find files ending in either .txt or .htm

If - else - fi format example

if [ ! -d datafolder ]

then

echo "Data folder NOT present"

else

echo "Data folder present"

-d for directory existence check, -f for files.
note that logical operators greater than can be represented as -gt and less than as -lt (less than or equal to as -le and greater than or equal to as -ge)

Output to a file

command > filename.txt e.g.

echo "ddd" > 3ds.txt

Output to the screen

Simply just display a file on the screen e.g. when outputting variables to it...

cat 3ds.txt

Output as the input to another command

Simply run the command on the data from the first all in one line as:

cat 3ds.txt | sed s/d/e/g

Here outputs "eee" to the screen.

Output the result of a command as arguments in another command

E.g. when you want to pass a filename instead of the actual output text use xargs instead of just the line "|" (pipe)

find 3ds* | xargs sed s/d/e/g | cat

Here since find just returns the filename without xargs we simply get 3es.txt. We even need to use xargs to view the file as:

find 3ds* | xargs cat

Sed (uses usually in-built sed program)

sed s/day/night/g input.txt > output.txt

replaces "day" with "night" when writing input file to new output file, g makes sure it is applied to all instances not just first.

or do the edits in-place:

sed -i s/day/night/ input.txt

We can also pass entire scripts to sed to run using the -f command:

sed -i input.txt -f script.sed

Get lines containing text string

grep searchterm textfile.txt

Simply returns the whole lines containing the searchterm

Simply cut text, use cut command

cut -c -10 textfile.txt

Execute a shell script using nice for prioritisation:

nice script.sh

must be executable (chmod +x script.sh)

Print the number of lines in a file

wc -l textfile.txt

also -c for bytes count and -m for character count

Tuesday, 3 June 2014

Run Commented and tabbed XMLs in Miind

Install xmlstarlet:

# sudo apt-get install xmlstarlet

Create a new BASH script:
# gedit miindx

Create BASH file in gedit:

#!/bin/bash
# Script to run commented XML files in Miind...
xmlstarlet c14n --without-comments $1.xml > $1_run_1.xml
cat $1_run_1.xml | tr -d '\t' > $1_run.xml
rm $1_run_1.xml
miind $1_run.xml

Make script executable:

# sudo chmod +x miindx

Run on test2.xml
# ./miindx test2

Adding paths to run commands permanently

I always manage to forget this so here is the code:

# sudo gedit ~/.bashrc

Add a line of the following to this file
# PATH=$PATH:/software/title/bin

restart terminal

Monday, 2 June 2014

How to install Miind on a new version of Linux Mint

How to install Miind on a new version of Linux Mint (version 17) with just Chromium installed so far (well also afs and kinit)

Notes for myself and anyone else.

Select the following packages in Synaptic (they will select dependencies too)
~ Install cmake
~ Install cvs
~ Install libboost1.55-all-dev
~ Install swig
~ Install python-dev
~ Install libroot-core-dev through synaptic
~ Install libroot-math-mathmore-dev through synaptic
~ Install libroot-graf2d-postscript-dev through synaptic
~ Install libgtkmm-2.4-dev
~ Install imagemagick
~ Install imagemagick++-dev
~ Install libmagickwand5
~ Install libroot-misc-table-dev
~ Install libroot-io-dev
~ Install root-system-bin
-- should now compile okay--
~ Install root-plugin-io-xml
~ Install root-plugin-hist-histpainter
~ Install root-plugin-graf2d-x11
I installed some other libroot, root-plugin packages that did not seem to affect running but play around with installing more of these if you need to!
Some of the above may install others in the list so some entries may be redundant...!

Install Miind using cmake (from terminal)
Get source from repo. (cvs as on website):
# cvs -d :pserver:anonymous@miind.cvs.sf.net:/cvsroot/miind co -P code in source directory ("code")
# mkdir build
# cd build
# cmake ..
# make
# sudo make install

Find the run file in the code folder in "build/apps/Miind/" (in terminal)
generate a simulation xml file:
# ./miind --g a.xml
run the simulation xml file:
# ./miind a.xml

Remember to remove "#" from terminal commands!