Dealing With .csv File Data By Name

I am grateful when I get a chance to have a a moment of productive programming;  wonder when the boom is going to be lowered whenever a period of more than a day of productive programming comes my way; and check my pulse if there is more than a week of productive programming. That has happened recently by needing to rework a Clojure program.

Recently, my Clojure and Python programming has been taking a decidedly more functional turn. That is write code that relies less on constructs from C or Java. In my case this was moving from database row positional dependence to referencing data by column name.

My work deals in those universal data transfer standards .csv and xml. The re-worked Clojure program in question is an application that compares two insurance reports. The information is  in .csv format, and each report comes complete with its own first row of column headers.

My original approach to this problem was positional, and after using clojure-csv to create a sequence of vectors, extracted data from each report row using nth. and column indexes defined as defs.

The code looked confusing, was hard to follow, and so I decided to create a sequence of maps using the saved column headers, transformed into map keys.

(defn gen-map-keys
  "Takes a sequence, and turns it into viable keys for a map.
   We opt to change spaces ' ' to dashes '-'."

  [in-seq]
  (map (fn [element] (keyword (cstr/replace element " " "-"))) (map #(cstr/trim %1) in-seq)))

Then each column header is zipped with a .csv row:

(defn gen-mapped-sos
    "Takes a sequence, from a sequence of sequences, and column keys,
     zipmaps them, and returns the map. The column keys must be sanitized,
     and already run through the key making process, before being submitted."

    [col-keys row]
    (zipmap col-keys row))

By doing things this way, functions became more generic, and easier to understand. This might have been due to the fact data comparison was taking place by map key, reworking the software, or a little of both, but the results were good.

Advertisements

Go ahead; throw an exception

Clojure, Python, and other modern programming languages lend themselves to the kind of code in which functions are called in all kinds of places, passed as parameters, returned from functions, and so on. In those instances, handling exceptions becomes tricky. It is not like the good old days of C/C++ coding, or at least today’s software seems less like those days.

Recently, while re-writing some Clojure code that depended on data extraction based on nth into functions that processed sequences of maps, I looked at the vulnerability of the new functions. They assumed the right data was being passed to them. I thought, well what if someone passes a sequences of vectors or lists?

The following functions show an example of what I needed and its solution. There is undoubtedly a better way to have done this, but at least this stops me dead in my tracks, so I can go fix a problem more easily.


(defn seq-of-maps?
  "Tests for a sequence of maps, and throws a custom exception if not."

  [s-o-m]
  (if-not (seq? s-o-m)
    (throw (IllegalArgumentException. "s-o-m is not a sequence"))
    (if-not (map? (first s-o-m))
      (throw (IllegalArgumentException. "s-o-m is not a sequence of maps."))
      true)))

(defn ret-map-col
  "Selects a key from a sequence of maps using a map key."

  [in-key in-seq]
  (if (seq-of-maps? in-seq)
    (map (fn [x] (in-key x)) in-seq)
    nil))


After The Clojure Course

I am ready to work on my final assignment for the Clojure Course. Overall, I give CodeLesson a thumbs up, but I will have comments on things that could be improved, should they desire them. In other words, if there is a survey, I will be filling it out.

Here are the highlights of what I’ve learned, and it’s not sequences, recursion, emulating objects, inter-operating with Java, or other technical concepts. Instead it is a new way to look at testing — it is just too easy to write tests not to — and ways of creating different layers of code logic, from which DSLs are created. Oh yes, and that means looking at where macros could be used.

I will take additional time to read the macro and functional programming chapters of Amit Rathore’s Introduction to Clojure, because there are some well written areas — the book very good overall — but some of the most complicated areas are just downright elegant. I will continue to look for Clojure articles, because I have found that even sources like the older edition of Stuart Halloway’s Programming Clojure has good pedagogy for particular topics.  Blogs like Alex Miller’s Pure Danger Tech have downright great tutorials, and I turned to some of these during the course, just so I could wrap my head around new concepts.

Finally, I need to start writing code and not worry where I am on the Clojure experience ladder. Instead, I will write tests, look for logical separation, and, no matter how simple the task, my experience tells me, you have to write code in and struggle with a computer programming language to achieve mastery in it.


An FP/Clojure Ah Hah! Moment

I have purchased either in e-book or printed form most of the major books on Clojure, including both from Manning Publications, Apress, O’Reilly and The Pragmatic Bookshelf . All these books are well written; all have slightly different approaches and styles.

Some are tutorial in nature, others are more like a tutorial/reference, and others contain philosophy in addition to tutorial. In some of these references, there is a sense you have programmed in Java previously, and I sense, and it is not directly implied, that the reader has dabbled in a Lisp dialect.

My Ah Hah! moment came recently when I realized that until a couple of years ago, the last languages I used in the commercial world were C++, Java 2, and Perl. Functional Programming (FP) was an academic subject, and though I had heard of Python, there was no pressing reason for me to learn it, at least back then. These books are written by authors with mostly a different programming learning and implementation environments.

So, as I work my way through Clojure In Action and Code Lesson’s Intro to Clojure, there are additional things for me to learn, the biggest of these the differing implementation environments and curriculum from when I started programming.

Here we go, then.


Manning’s Clojure in Action

Many of you may already have Amit Rathore’s Clojure in Action through Manning’s MEAP program, but good news for non-MEAP people, like me, Amazon has already indicated they are shipping me Clojure in Action soon. That means Code Lesson’s Clojure course should start soon.


More Clojure Training

As I await the release of Amit Rathore’s Clojure In Action and the subsequent on-line course on CodeLesson, I am out beating the bushes for more Clojure training, and I found labrepl.  I wound up installing lab_repl, and using cake repl to issue these startup commands:


(require 'labrepl)

(labrepl/-main)

Then, I went to localhost:8080 to use the labrepl web application.

There are instructions to install into Eclipse http://code.google.com/p/counterclockwise/.

This tutorial comes from Relevance, who teach Clojure.


Clojure Training / Clojure Koans

I am not the first person to start learning Clojure, who is looking for tutorial-level material, and I will not be the last. My programming experience goes back thirty years, but all of that time has been using so-called imperative programming languages like PL/I, C, C++, VB/VBA/VBScript, and yes, a little bit of Java. Learning those languages both formally in the classroom and from the more experienced people in our environments could not prepare us for functional programming.

I have been using the 4Clojure web site, but found some problems marked elementary or simple seemed the opposite. I have tended to treat it as a game rather than a learning experience. I had seen Clojure Koans mentioned in Clojure Google Groups and on blogs, but never investigated it … until today.

I recommend it.