Does Clojure Tend To Encourage Small, Efficient Functions?

I have only two small Clojure projects, and other than writing tools on which nothing major depends, these projects will probably be the only ones in my current position. The rest will be done in Web languages and Perl. So, I enjoy a chance to enhance the Clojure projects.

I do not know why, and am not aware of any conscious prejudice, but writing Clojure code encourages me to create small functions, and external Clojure projects, like a utility project containing various odds and ends to handle sequences, time/date manipulation, and other useful stuff.

I have written enough Perl and Python code to appreciate what those languages do, but, for some reason, my functions tend to be longer and wider in definition than my Clojure functions.


Less Is More, But You Probably Already Knew That

I remember my first introduction to Unix. It was at Harvard Extension in a Programming C in Unix course.  I don’t even believe Minix had been invented yet, let alone Linux. Basically, my introduction to programming in Unix involved writing small, tight, do-one-thing-well programs — the programs assumed to be written in C –  and tie those programs together with shell scripts. How true all that was, but appreciating its realization was to take nearly my entire career.

As I now maintain a two-year old Clojure program and a four-year old water AMR system. written in Python, it seems the choice of language is not what matters, but how everything was designed and implemented. It doesn’t hurt that Python encourages simple, clean function design and the creation and inclusion of modules, or that Clojure and leiningen  highly encourage module design using projects, with their resulting name spaces and underlying directories. However, it all seems to be boil down to how I was taught to implement logic more than anything else.


Finding A Less Risky Way To Use Clojure

My position represents either a sub-department or department of one. There is only one person in our municipality doing my kind of work, and a very knowledgeable consultant works with me. As my retirement looms in the next few years, I do not want to be known as someone who left a tangled mess. After all, more than a decade ago, I inherited just that because of the untimely passing of my predecessor, another one-person band. The mess was undocumented, uncommented Informix 4GL source code, virtually no build, no source code control, and no systems documentation.

The consultant with whom I work wants to standardize on the implementation language we use, and I agree with him. We haven’t chosen Perl because we think it is the best, but because a lot of people know it, and it works well for standalone and web applications. In our opinion, it doesn’t beat PHP for the web, but in a dual role works well for CGI and command-line, unattended applications.

After learning Clojure, I would rather write in it, even after having invested a store/forward configuration/meter reads system in Python. But, there is that trying to standardize thing I mentioned previously. So, I am going to take the advice I saw on one of the bulletin boards to someone just starting out. Write something and show it.

What I write will be tools on which someone coming into my job won’t have to maintain or reproduce, at least at the start. The rest of it, including a MySQL database crawler to keep the number of daily reads manageable at 14M rows of data, will be done in Perl. All of the people who said Clojure would affect my thinking even if I only learned it and never used it much were not kidding. My problem with Clojure is having to use something else. I guess that is a good thing for Clojure.

 


Thoughts of Immutable and Partially Immutable Programming Languages

This post isn’t about language bashing. If you inherit a large body of code, say for a tax billing system, you realize you won’t be re-writing any of that code anytime soon, especially if you don’t know RECAP from January 1 Owners or Excise Service Warrants from Urgent Parking Notices.

Because it was here to stay, at least for a while, I learned to like Informix 4GL. I have pushed the language to its limits, including blowing the stack, evidenced by cursors never reaching the end of their working sets. I prefer other languages, like Clojure, Perl, and Python, but we’re making a cost-saving decision now to go back to well-worn and tested 4GL programs for generating water liens (for unpaid water bills) rather than re-writing everything in a language we deem “better”.

Instead, this is post is about the musings of Scala and Python, and how one achieves as close to pure functional programming with these languages. I have written a major project in Python, and know a reasonable amount about the language. I’ve had only a slight peak at the Scala for the Impatient (a very good read by the way), and am by no means an expert, let alone even knowledgeable.  With Haskell and Clojure, your variables are always immutable. In Clojure you could defeat immutability by using and modifying a  ref. 

Can you achieve enough immutability with tuples in Python for it to be considered functional, and whether or not you can achieve immutability enforcing the use of

 val 

in Scala programs? Over the years, I have worked for well-run development groups, and despite good management and team interaction, not everything could be enforced, especially with coding style, and definitely when defining objects when C++ first began to be a viable programming language.

I will continue to dabble in Clojure, write major new programs in Perl, and maintain the old 4GL software, and somewhere keep thinking about immutability.


A Simple Clojure Program To Read an Informix Database

This post assumes you have Informix properly installed and have access to database tools like dbaccess. My configuration is running on CentOS 6.5 32-bit. It also assumes you can get the Informix JDBC driver installed into maven.

From the day I got my hands on The Joy of Clojure and eventually other excellent Clojure books, I have wanted to use Clojure to access an Informix database. In our case, it is an Informix SE database running on Linux.

This is what I did to get it running, with help from these stackoverflow questions and IBM Support. Although my first stackoverflow question involved korma, it made sense to keep things simple, and just use the Informix JDBC driver directly.

1) For SE, the latest support jdbc driver is 3.50.JC9, so that is what I installed.

2) You need to create a database. I created the stores7 directory and stores7.dbs. The demos I used all require that a database exist or be created.

3) Simple Java file installed at /opt/IBM/Informix_JDBC_Driver/demo/basic/SimpleSelect.java that we modified in a local directory, rebuilt, and ran


/**************************************************************************
*
* Licensed Materials - Property of IBM Corporation
*
* Restricted Materials of IBM Corporation
*
* IBM Informix JDBC Driver
* (c) Copyright IBM Corporation 1998, 2004 All rights reserved.
*
****************************************************************************/
/***************************************************************************
* Title: SimpleSelect.java
*
* Description: Demo a simple select operation
*
* An example of running the program:
*
* java SimpleSelect
* 'jdbc:informix-sqli://myhost:1533:informixserver=myserver;user=;password='
*
* Expected result:
*
* >>>Simple Select Statement test.
* URL = "jdbc:informix-sqli://myhost:1533/testDB:informixserver=myserver;user=;password="
* Select: column a = 11
* >>>End of Simple Select Statement test.
*
***************************************************************************
*/

import java.sql.*;
import java.util.*;

public class SimpleSelect {

public static void main(String[] args)
{
     String newUrl = "";

     newUrl = "jdbc:informix-sqli://steamboy:1498/testdb:informixserver=steamboy;usev5server=1;DBPATH=/home/cnorton/stores7";

     Connection conn = null;
     int rc;
     String cmd=null;
     Statement stmt = null;

     String testName = "Simple Select Statement";

     System.out.println(">>>" + testName + " test.");
     System.out.println("URL = \"" + newUrl + "\"");

     try
     {
          Class.forName("com.informix.jdbc.IfxDriver");
     }
     catch (Exception e)
     {
          System.out.println("FAILED: failed to load Informix JDBC driver.");
     }

     try
     {
          conn = DriverManager.getConnection(newUrl);
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: failed to connect!");
     }

     // Drop table before starting - ignore errors
     try
     {
          Statement dstmt = conn.createStatement();
          dstmt.executeUpdate("drop table x");
     }
     catch (SQLException e)
     { ; }
     try
     {
          stmt = conn.createStatement();
          cmd = "create table x (a smallint);";
          rc = stmt.executeUpdate(cmd);
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: execution failed - statement: " + cmd);
          System.out.println("FAILED: " + e.getMessage());
     }

     try
     {
          cmd = "insert into x values (11);";
          rc = stmt.executeUpdate(cmd);
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: execution failed - statement: " + cmd);
          System.out.println("FAILED: " + e.getMessage());
     }

     try
     {
          cmd = "insert into x values (22);";
          rc = stmt.executeUpdate(cmd);
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: execution failed - statement: " + cmd);
          System.out.println("FAILED: " + e.getMessage());
     }

     // INFORMIX_EXTEXT_BEGIN Simple1.jav
     try
     {
          PreparedStatement pstmt = conn.prepareStatement("Select * from x "+ "where a = ?;");
          pstmt.setInt(1, 11);
          ResultSet r = pstmt.executeQuery();

          while(r.next())
          {
                 short i = r.getShort(1);

                 // verify result
                 if (i != 11)
                        System.out.println("FAILED: Expected = 11 Returned = " + i);
                 else
                        System.out.println("Select: column a = " + i);
          }
          r.close();
          pstmt.close();
     }
     catch (SQLException e)
     {
         System.out.println("FAILED: Fetch statement failed: " + e.getMessage());
     }
     // INFORMIX_EXTEXT_END Simple1.jav

     try
     {
          cmd = "drop table x";
          rc = stmt.executeUpdate(cmd);
          stmt.close();
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: execution failed - statement: " + cmd);
          System.out.println("FAILED: " + e.getMessage());
     }

     try
     {
          conn.close();
     }
     catch (SQLException e)
     {
          System.out.println("FAILED: failed to close the connection!");
     }

     System.out.println(">>>End of " + testName + " test.");
   }
}

Here is the Clojure project.clj


(defproject db-test "0.1.0-SNAPSHOT"
     :description "Clojure database test"
     :url "http://example.com/FIXME"
     :license {:name "Eclipse Public License"
     :url "http://www.eclipse.org/legal/epl-v10.html"}
     :dependencies [[org.clojure/clojure "1.5.1"]
                    [org.clojure/tools.cli "0.1.0"]
                    [com.informix.jdbc/com.springsource.com.informix.jdbc "3.0.0.JC3"]
                    [org.clojure/java.jdbc "0.3.3"]]
     :repositories [["springsource-release" "http://repository.springsource.com/maven/bundles/release"]
                    ["springsource-external" "http://repository.springsource.com/maven/bundles/external"]]
     :main db-test.core
     :aot [db-test.core])

And here is the Clojure core.clj. test file.


(ns db-test.core
    (require [clojure.string :as str])
    (require [clojure.java.jdbc :as j])
    (:use [clojure.tools.cli])
    (:import java.util.Date)
    (:gen-class))

; Parses for options passed in on the command line.

(def if_SE_engine_type "SE")
(def if_SE_absolute_log_path "/home/cnorton/stores7/stores7.log")

(defn parse-opts
     "Using the newer cli library, parses command line args."
     [args]
     (cli args
      (required ["-host" "Informix host"] )
      (required ["-server" "Informix server"])
      (required ["-dbpath" "Full path to database directory"])
      (optional ["-port" "Informix host's service port" :default "1498"] )
      (optional ["-username" "user name"] )
      (optional ["-password" "password"] )
      (optional ["-database" "Informix host's database" :default "stores7/testdb.dbs" ] )))

(defn -main
       [& args]
       (if (= 0 (count args))
          (println "Usage: db-test -host  [-port ] -database  -dbpath  -username  -password  -server ")

       (let [opts (parse-opts args)
                   start-time (str (Date.))]

            (def informix-db {:classname "com.informix.jdbc.IfxDriver"
                              :subprotocol "informix-sqli"
                              :subname (format "//%s:%s/%s:informixserver=%s;usev5server=1;DBPATH=%s"
                                         (:host opts)
                                         (:port opts)
                                         (:database opts)
                                         (:server opts)
                                         (:dbpath opts))})

            (let [customer-list
                  (j/query informix-db
                  ["select * from customer"])]

                  (doseq [customer customer-list]
                      (println customer))))))

Why Moving Special Casing To The Data Can Be A Good Thing

From June 2013 through mid-February 2014, I have worked almost exclusively on rewriting our town’s water billing system. It works in conjunctions with our AMR system. That system  serves as a store and forward mechanism, which sends meter configuration changes to our hosted application, and also collects daily reads for every water meter in our “district”.

We chose Perl for the implementation language, but even more important than the implementation language were the choices we made to avoid special casing and to make things flexible. Here is one example, a meter that measures the amount of water your home or business uses, is completely different from a fire service line. A fire service line is required to provide a separate source of water to fire suppression sprinklers.

Despite these differences, our software treats these two entities similarly. Every meter has a row in the meter table, and every fire service also has a row in the meter table. With this, software can go look up the charge for a fire service line, just as if that line were a meter. This avoided special casing in the main software, and, what I’ve found over the years is when data can be treated similarly, there are fewer checks and things that can go wrong in software. One way of looking at this is the special casing moves from conditional testing in software to the data itself.

In the case of a meter or a fire service, the data contains the decision-making. A column in the meter table contains a value, that if not present means the data represents a meter. If a known value is present, the data represents a fire service line. There are fewer checks in the software.

This isn’t a unique discovery. I certainly didn’t invent it. It just seems hard to practice when you are under a tight deadline for a project, but it seems to pay off handsomely at the end.


The Most Important Aspect Of A Programming Language*

*when used for a municipal programming project

Since June of 2013, I have been working on re-writing my town’s water billing system. This project has included rewriting part of the meter configuration and water billing systems, originally written using Informix 4GL. So this project has included writing new software in Perl and re-writing existing software in Informix 4GL.

The greatest aspect of using Perl — and I can see parallels in Clojure lists, vectors, and maps — is Perl hashes are very flexible. Flexibility is very important in a municipal project. Requirements tend to be come in on the fly, and water consumption and billing rules are certainly not, as folks like to say rocket science, but there are a lot of rules, like what happens when a water meter’s digits rollover to 0000, or you replace a meter and need to calculate consumption based on the old and new meters’ reads. And, most of these rules are documented in one place, the software. And you can only read those rules, if you own the software.

Your programming language needs flexibility, like the ability to cache away a meter reading value that will be displayed on a bill, as opposed to a meter reading value that can be subtracted from another value in which a meter rollover may or may not have occurred. As we now get ready to approve and mail out the first water bills with 3-tier rates, billing for fire service lines (if you have one), as well as the usual administration charge, I credit the [almost] success of the project to the ability to be flexible and modular in our programming language.


Follow

Get every new post delivered to your Inbox.