Schubert's Blog

A biker and technology enthusiast's
view of the world

Zipping with C#

In order to save on bandwidth and improve the user experience, by making the route files faster to download, I needed a way to compress files. This was one of the goals for Mapze Beta 1 but considering future improvement, I wanted Mapze.com to be able to uncompress uploaded route files too. Allowing users to upload GPX, ITN, CSV or multiple route files by zipping them. Mapze.com would than unzip the route files on server and process each file like normal route files.

The soution : the GZipStream class

The .Net Framework comes with the System.IO.Compression namespace which has an interesting class called GZipStream. At first I though job done. The namespace and class name clearly indicated a class that can handle compressing and uncompressing streams. Plus, using a class in the .Net Framework has the advantage of knowing it will be supported in future releases, if not, an alternative will be provided. It will be installed with the Framework, so one less assembly to be concerned with at deployment time and if there are any bugs or issues, Microsoft will take care of it.

Turns out the GZipStream class can only handle .gz files, well known formats like zip and tar are not support. Another important point was it did not support hierarchical or directory level compression, or at least did not make the task easy. I needed the ability to read and create zip files that could contain multiple files or at least files at one directory level deep. Considering the above two points, it was time to search for other solutions. A quick google brought up sharpziplib library. It supported Zip, BZip2 and GZip format and was written in C# for the .Net Framework. Bliss.

The solution: the SharpZipLib

The final zipping code snippet looks like:

bool GetZippedITNFile (string SaveToFilePath, int ITNFileCount)
{
//Create a Zip file
ZipOutputStream zipstream = new ZipOutputStream(File.Create(SaveToFilePath));
//Set the compression level
zipstream.SetLevel(9);

//Temporary variable to indicate a new zip file entry
ZipEntry tempZipEntry;
//Temporary variable to store the file contents
string tempITNFileContents;
//Temporary variable to encode the contents before adding it to the zip stream
UTF8Encoding enc = new UTF8Encoding();

for(int currentFileNo = 1 ; currentFileNo <= ITNFileCount; currentFileNo++)
{
//Create a zip entry and set its filename, i.e. Route-1.itn, Route-2.itn, etc
tempZipEntry = new ZipEntry(string.Format("Route-{0}.itn", currentFileNo));
//Set other file properties
tempZipEntry.DateTime = DateTime.Now;
tempZipEntry.Comment = string.Format("Route file {0}", currentFileNo);

//Add the zip entry to the stream. Indicates that any
// characters / bytes written to the stream are now a part of a new zip file
zipstream.PutNewEntry(tempZipEntry)
//Get the file contents to zip
tempITNFileContents = GetITNFileContents(currentFileNo);
//UTF8 encode the contents and add it to the zip stream
zipstream.Write(enc.getBytes(tempITNFileContents), 0, tempITNFileContents.Length);
}

//Flush any buffered contents to the file
zipstream.Flush();
//Close the file
zipstream.Close();
}

The out come will be a .ZIP file that can be opened with well known zipping softwares like WinRAR, WinZip, 7Zip, etc. The .Zip file will containing .ITN files and the ITN file will be named Route-1.itn, Route-2.itn, Route-3.itn, etc

By Schubert on 03 January 2010 22:52

Categories: Development | Tags: , , , ,

Permalink | Comments (0)

Submit to DotNetKicks...

Mapze.com Beta 1

I wanted a challenge, a website to work on from ideas and concepts, all the way to deploying the final product on an internet accessible server and getting public feedback.

The solution: Mapze.com

Mapze.com is an event sharing website, to keep it short and simple. Although, its much more than that. The first Beta focused on bikers like myself. Besides the usual register, update my details and update my password page, the user should be able to plan a route online using Google Maps and upload a route in format like GPX, ITN, CSV, etc. They should also be able to download the routes in various GPS friendly formats (at least the well know devices) and share their planned rideout dates, details and route experiences in one central location. All very interesting, as I've never developed any of the above from scratch or seen anything like it on the web.

Technologies of choice: having worked on ASP.Net for over 3 year now, I wanted the developement process to be a challenge too. The ASP.Net MVC web framework being new and promising, served the purpose well. It allowed for cleaner SEO friendly urls and gave me full control over the HTML, prefect for learning some CSS at the same time. Having more exposure to the C# language over VB.Net, I decided to use C# 3.0 and taking advantage of the new LINQ To SQL ORM tool for the database access layer. Since Mapze.com needed to save GPS locations and needed to do searches and calculation based on global co-ordinate, like find rideouts planned within 5 miles of London, I decided to take advantage of the new spartial data types and function in MSSQL 2008.

The above technologies formed the base of Mapze Beta 1. The other technologies involved were Google Maps, so users could plan and amend their routes online. JQuery and Javascript to handle AJAX calls to google geo services and presenting them on the map. XML Parsing for reading GPX (XML) files. NUnit and Rhino mocks to follow the TDD style of progamming. MSSQL Full text indexing to search through event keywords. .Net Multilingual support, to make the website accessible in various language. Elmah and Log4Net to gracefully handle any error and warning on the website. DiscountASP.Net for hosting the website. And finally RapidSSL for the SSL certificate, for when users login and update their password.

Like most project I've worked on for clients and personal ones, I came away with some interesting lesson and a revised list of Do's and Don't's for future projects. These are a few lesson learnt from the first Beta:

  • Setting up SSL on IIS was a lot easier than I had initally though
  • CSS is a powerful display formatting language
  • Route formats are inconsistant, GPX being the most flexible and feature rich format
  • the .net build in Zipping classes are good, but ... library is better

Taking on the full software development life-cycle for even a simple concept website like Mapze.com, definitely involved a lot of creativity, architectural and scability considerations ... and a lot of hard work.

Screen shots of the final product:

Mapze.com Beta 2 is live! Click here for more ...