Zhikai Wang Homepage

Heteroclinic.net logo

www.heteroclinic.net

	Tiny CSV Reader I. the requirements 201506 CSV files are handy for bulk processing and it is also human readable. Wikipedia has an article about the formalization, we quote here as the following: MS-DOS-style lines that end with (CR/LF) characters (optional for the last line) An optional header record (there is no sure way to detect whether it is present, so care is required when importing). Each record "should" contain the same number of comma-separated fields. Any field may be quoted (with double quotes). Fields containing a line-break, double-quote, and/or commas should be quoted. (If they are not, the file will likely be impossible to process correctly). A (double) quote character in a field must be represented by two (double) quote characters. https://en.wikipedia.org/wiki/Comma-separated_values last retrieved June,2015.
	Tiny CSV Reader II. the Nature Language Expression 201506 It is important to identify or count the appearance of double quote literal. Here we shorten it to quote. If quotes appear continuously in odd times, the first quote it is a field border. A record separator can be a new line literal. Before it, the field borders must appear even number of times, otherwise it is a literal inside a field. Namely a record separator is a new line with "quotes appearing continuously in odd times" appears even times before it. Further, the next valid new line as a record separator has always even number count of quotes before it. A fields separator can be a comma literal. Same as the record separator, "quotes appearing continuously in odd times" appears even times before it always. A field can be an empty String. A record can be an empty line. A whole file can be empty. If we don't handle them, we may halt the program before an expected exit/end.
	Tiny CSV Reader III. Transfer the Nature Language Expression to Programmable Regular Expression 201506 Here after a long hike in a winding trail, we implement with Java for convenience package net.heteroclinic.tinycsv; import java.io.File; import java.io.FileNotFoundException; import java.util.Scanner; import java.util.regex.Matcher; import java.util.regex.Pattern; /** * <p/> * <p/> * Tiny CSV Reader * <p>Third party licenses/rights are automatically cascaded.</p> * Given you keep this header, you can do anything with this program. * @author Zhikai Wang/www.heteroclinic.net * Reference " https://en.wikipedia.org/wiki/Comma-separated_values last retrieved June,2015. / public class TinyCSVReader { public static void main(String[] args) throws FileNotFoundException { { // Need copy test/resources files to main/resources folder final String data = new Scanner(new File(TinyCSVReader.class.getClassLoader().getResource("testfile2").getFile())).useDelimiter("\\Z").next(); System.out.println("=== 1 ==="); // the or conditions should be mutual exclude or the parser may be confused. maybe that is former cases are not expected. Pattern patternForRecord = Pattern.compile("(?s)" + "(" + "(\\G" + "([^\\\"]?)" + "([\\\n]\|$))" + "\|" + "(\\G" + "((([^\\\n]?)[\\\"](.?)[\\\"]([^\\\n]?))+?)" + "([\\\n]\|$))" + ")" ); Pattern patternForField = Pattern.compile("(?s)" + "(" + "(\\G" + "([^\\\"]?)" + "([,]\|$))" + "\|" + "(\\G" + "((([^,]?)[\\\"](.?)[\\\"]([^,]*?))+?)" + "([,]\|$))" + ")" ); Matcher m = patternForRecord.matcher(data); while (m.find()) { String record = data.substring(m.start(), m.end()); if (record.trim().length() <= 0) continue; // replace the trailing record = record.replaceAll("\n$", ""); //System.out.println(record); Matcher m2 = patternForField.matcher(record); System.out.print("\|"); recordLoop: while (m2.find()) { String field = record.substring(m2.start(), m2.end()); // replace the trailing // if field does not have a comma at the end, it is the last field if ( field.length() > 0 && (field.lastIndexOf(",") == field.length()-1)) { field = field.replaceAll(",$", ""); System.out.print(field+"\|"); } else { System.out.print(field+"\|"); break recordLoop; } } System.out.println(); System.out.println("----------"); } } } }
	Tiny CSV Reader IV. The Proof of Concept Test File 201506 We post the test data file here. It provides limited scenarioes that in real life could make a lesser program fail. a1,b1 b1,c1," d1 d1 " a2,"b2 b2","""c1"""," d2 " a3,"b3 "" b3","""c3""",d3 a4,b4,c4, ,b5,,d5 a6,b6 b6,c6," d6 d6 " a7,"b7 b7","""c7"""," d7 " a8,"b8 "" b8","""c8""",d8 a9,b9,c9, ,b10,,d10