Testing cron Jobs

1. Use crontab -e when possible when editing your cron table. It validates the entries on save.

2. Redirect output to a log file. For example:

 * * * * *  /path/script.sh &> /path/script.log    

3. Run your script manually under the same user used in crontab.

4. If you are using run-parts (cron.daily, cron.hourly, etc), you can manually run run-parts.

run-parts /etc/cron.daily    

5. If you are not using run-parts and putting cron jobs directly in crontab, you can change the cron schedule. For example, instead of running the job daily, run it every minute for testing purposes.

6. Monitor the cron log file. Here is tailing the log in CentOS.

tail -f /var/log/cron    

7. Here are many possible reasons why crontab does not work.

http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work


Installing ElasticSearch with River JDBC on RedHat/CentOS

1. Install OpenJDK if it doesn’t already exist. Here we are running as root, so sudo is not required.

> yum install java-1.7.0-openjdk.x86_64

2. Install ElasticSearch using yum. The latest ElasticSearch version is 1.4 at the time of this writing.

> rpm --import https://packages.elasticsearch.org/GPG-KEY-elasticsearch
> vi /etc/yum.repos.d/elasticsearch.repo

# contents of elasticsearch.repo
[elasticsearch-1.4]
name=Elasticsearch repository for 1.4.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.4/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1

> yum install elasticsearch

3. Start ElasticSearch as a service, and tail the log.

> chkconfig --add elasticsearch
> service elasticsearch start
> tail -f /var/log/elasticsearch/elasticsearch.log

4. Install River JDBC. We are using the latest Rivers JDBC version, which is 1.4.0.8 at the time of this writing.

> cd /usr/share/elasticsearch
> bin/plugin --install jdbc --url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.4.0.8/elasticsearch-river-jdbc-1.4.0.8-plugin.zip
cd plugins/jdbc/
> wget {url-to-jdbc-jar}/ojdbc6.jar
> service elasticsearch restart

5. Configure River. Do the initial, oneshot data load from Oracle. Replace the variables which are surrounded by {}. If you are unfamiliar with ElasticSearch indexes and types, they are similar to relational databases and tables.

PUT
    {
        "type" : "jdbc",
        "jdbc" : {
            "url" : "jdbc:oracle:thin:@//{IP}:1521/{SERVICE}",
            "user" : "{USERNAME}",
            "password" : "{PASSWORD}",
            "sql" : [{
                "statement" : "{QUERY}"
            }],
            "index" : "{INDEX}",
            "type" :  "{TYPE}"
        }
    }

6. Load new or updated data every 10 minutes.

PUT
    {
        "type" : "jdbc",
        "schedule" : "0 */10 * * * ?",
        "jdbc" : {
            "url" : "jdbc:oracle:thin:@//{IP}:1521/{SERVICE}",
            "user" : "{USERNAME}",
            "password" : "{PASSWORD}",
            "sql" : [{
                "statement" : "{QUERY where created_on > ? or modified_on > ?}",
                "parameter" : [ "$river.state.last_active_begin", "$river.state.last_active_begin" ]
            }],
            "index" : "{INDEX}",
            "type" :  "{TYPE}"
        }
    }

SOCKS Proxy over SSH on Windows using Putty

Here’s how you can setup a local SOCKS Proxy over SSH on Windows. Here we are using Putty.

1. Create a new Putty SSH session.

New Putty SSH Session

New Putty SSH Session

2. Setup the SSH tunnel. Add a source port with a dynamic destination. Here we are using the default SOCKS port 1080.

SSH Tunnel

SSH Tunnel

3. To change your local network settings, go to Internet Properties and enable the proxy server setting.

Enable Proxy Server Setting

Enable Proxy Server Setting

4. If you are a Firefox user, you can change just the Firefox connection settings.

 

Firefox Connection Settings

Firefox Connection Settings

5. If you have Java code that needs to go through the SOCKS proxy, you can set the following System properties.

System.getProperties().put("socksProxyHost", "localhost");
System.getProperties().put("socksProxyPort", "1080");