Julia performance - avoid global variables as per performance docs

I'm still finding that get significant speedups for all tasks in switching from Perl  to Julia
But it's quite easy to write code that is orders of magnitude slower if you write it in the wrong way, so if the timing matters it's essential to read the documentation on performance
e.g. here's some simple code that strips ascii nuls out of a text file


fileinname = "Log.txt"
filein = open(fileinname)
logfile = open(string(splitext(fileinname)[1],".PSV"), "w+")

# read and write
buffersize = 65536
#b::Vector{UInt8}
b = zeros(Uint8, buffersize)
readsize = buffersize
while readsize == buffersize
readsize = readbytes!(filein, b, buffersize)
#replace null with space
for i in 1:readsize
b[i] == 0 && (b[i] = 32)
end
    write(logfile, b[1:readsize])
end
It takes about 42 seconds to run on a 80 MB text file (windows 7 julia 0.3.3)

perl -pe "tr/\000/ /;$_" log.txt > log.DLF

takes about 4 seconds

But eliminating the global variables by merely putting a
let 
at the beginning and
end
 at the end of code takes the time down to 0.65 seconds, and @inbounds knocks another fraction of a second off.

b[1:readsize] apparently makes a copy of the data, but 0.4 is I think due just to refer to the existing data.

SQLite and Julia

I've found SQLite a good way to store whole lots of local data and even config information for applications. For config information it provide a clean interface for any required updating programmatically, with tools like SQLite manager as an add in it's easy to modify and easier to keep formats correct than with a text editor and xml or the more plausible yaml. Having a single file on the disk is also nice. I've not had a problem with corruption and trying to recover from it yet.
The one hitch is version control - yaml and git works well and is simpler than comparing changes with

sqlite3 somedb.sqlite .dump > somedb.txt

and using backups.

Julia has an easy to use and working library named not surprisingly SQLite
An update was incompatible with 0.3 Julia - before I'd sorted a work around, the maintainer Jacob Quinn had fixed it.
I'm still astounded that it's possible to come across salesman for enterprise software trying to spread FUD on open source software in between trying to convince me that JSON was something to do with Java, and vomiting out a torrent of technology buzzwords. This is when almost without exception I've had better response times on fixes for open source software I use,  even where we don't have any maintenance agreements, than on any closed source software, even when the supplier is cooperative and we do have maintenance agreements.

Ebola Statistics

Update
Wikipedia has been updated with similar and better graphs

looking at
http://ebolastats.info/  I couldn't see where things were getting worse better or whatever
Picking up the stats from Wikipedia and scraping them quickly (and there are lots of ifs in the stats)
And then plotting them with Mathematica on log plots gives the plots below.
It's not a heartening outlook at the moment, heading onto the level of impact of the Spanish flu next year if nothing changes for the better (that's the simple depressing plot at the bottom)
Note that the numbers are supposed to be an underestimate, there will be lots of deaths from other causes as health systems fail and there's always the risk it becomes more infectious. Against that the drugs may work or other approaches such as innoculating with the blood of survivors, and resources from richer countries are being put into it.
Nigeria's outbreak appears under control which with a potential 180 million people is good news.
In Guinea, 12 million people, it seems to be steadily growing.
Liberia, 4 million there's a hint the rate of increase is slowing
Sierra Leone, 6 million the apparent lower death rate may only be due to the later arrival of the disease and the month for it to be fatal, i.e. the deaths lag the cases.

And some gloomy perspective with a  static picture
Annual malaria deaths - something like 600,000
Syrian civil war toll ~200,000 deaths, 22 million people
Ukraine ~ 4000
Ebola ~ 4000
Camel Flu ~ 320 (40% of those infected as another comparison)
US car deaths 34,000 (2012)





Cumulative death rate







Simple extrapolation