p Mike Hall's Shareware

What's new?

08/12/14


In General
A little bit on Online learning. Also revisiting a couple past efforts. Putting some more time into the Blackjack code, a household request. Some more code that was left a little bit of a mess. So a lot of half thought out features I was just starting to add and some debug messages if console is enabled for anyone

I might start including source on that, soon, in case anyone wants to double check me on anything. There are a couple strange results in running the Simulator for one thing. Basic Strategy usually tends to result with up finishes in the long run. My understanding is with normal Las Vegas rules it should lose money. I don't think Vegas would be really running rules that allow the player to win without even card counting. It also beats Hi-Lo lite which does involve card counting and should be better. So there is probably something wrong that I, or someone else? should figure out sometime. Anyhow these things are a bit involved and there might be any number of errors. Providing the source if anyone wants to check seems to make sense.

If I haven't mentioned the Simulator before I think it is a little fun to play around with.

The other re-visited code is TRZ to see if I can figure out the kqueue nio.2 WatchService freeze. About all I have to say on that unless I make some progress

Also more time into the Kaggle Forest competition, still messing with Weka meta wrappers. Not too much in real interesting progress so far.

Online learning. Back to taking a couple of courses. Coursera - From the Big Bang to Dark Energy
Also going through a recommended Neural Network course...
Neural networks class - Université de Sherbrooke
I'm also going through the Data Camp Introduction to R, to keep that from getting too stale.

List of courses taken

Machine Learning Weka and Kaggle and anything else Machine Learning related

HalfPipe

Update 04/20/14

Also haven't been putting as much time into this as I'd meant to. I do have some simple assignmnet code in place.

Like s = "Hello, World!" or x = 10. Next would be like java_tasks = `exec jps`. I'm figuring the Unix like back ticks right now. I've done similar for the Ruby and Javascript interfaces using ArrayCmdOut to receive the output on calls back into the shell.

Sort of wondering how to handle the arrays right now. More java like or more R like. Have a R vector like idea?

Then I still need to handle objects and interface to Weka/R. Anyhow putting up the current application with assignment as it is. Might spew debug messages yet as well for those.

HalfPipe

Current Weka interface information can be found on the Machine Learning page added above. Information on old related changes can now be found at the Past Updates page.

AppleScriptEngine

Besides that, for a change. I do have a new Mac. Thank you. Yes, very nice. I didn't get around to trying my HalfPipe application for a while. When I did, it wouldn't launch. The app has a launch dependency on the JSR 223 AppleScript interface updating running task information. This should probably be lazy loaded when needed and not at launch but that's another bug. The interface didn't seem to work, the app crashed NPE. Something in my new configuration seems to have uncovered a miss in the openjdk configuration for this. Possibly something disappeared in the Apple java stuff, anyhow they don't have to config right and it started crashing my app.

This stackoverlow link got me on the right track. Although it indicated the config is the jre/jdk rt.jar file. When I changed this the AppleScript interface did start turning up in test but didn't with the 'jrunscript -q' command. This post on the macosx-port list indicated the correct configuration file is in resources.jar. Changing that also got the jrunscript command showing AppleScript. I filed a bug report that was moved to the openjdk JIRA.

Now, some this legacy(?) Apple stuff doesn't seem entirely at home in the openjdk effort. I have already moved some of the file related to github already along with my trz project. There I had even added a FileHelper class to go with the Apple derived FileManager class to show that we, the Mac Java community, might be able to even extend this stuff if we open source support it.

At least for my application, it seemed to make sense to have this interface available dylib/jar instead of expecting any users to make changes to the JDK/JRE configuration. So I have also added that out to github as AppleScriptEngine. It pretty just includes the pertinent parts, the built jar/dylib and a test class. The test class lists available engines, should inlude AppleScriptEngine, and test runs a AppleScript. I used my pick card test. Although AppleScript is mostly supposed to be about inter-process communication I usually try to avoid test cases that involve any other applications. My other usual choice is using the Zarvox voice to say Hello, World! If anyone would prefer that, let me know.

R

While I was on the JSR 223 thing I decided to go ahead and finish adding the one for R I had looked at before. That is now included. in HalfPipe. There can only be one unique instance of R running so that interface keeps a static reference.

That makes use something like...
us.hall.scripting.RS
t = 3
class(t)
[CTRL+D to terminate System.in input]
numeric
echo this
this
[So we are clearly no longer inputing to R now right?]
us.hall.scripting.RS
class(t)
numeric

The basic idea is that the t variable is still set to the same value. One persistent instance.

Give me back my debug println's

I had written something for this when I heard on the java-dev forum that System.out.println's no longer went to Console. I had use for it in debugging the R interface. So, I have added it to the java i have on github

Engines

So, that now brings HalfPipe to supporting five JSR 223 interfaces. I thought a little bit about that would be an idea. You can, of course, see what they are with the engines command. The output of that is shown here.

Briefly, what they are and what they are good for...

Mozilla Rhino (1.7 release 3 PRERELEASE) The builtin javascript. Supported by openjdk. Basically Rhino I think.

AppleScriptEngine (1.1) The Apple donated default. My application still has dependencies including launch for one.

JSR 223 JRuby Engine (1.7.4) JRuby. I did a rather basic FFI interface to the ObjectiveC runtime using this. I was thinking this would be the one implementation of that I would do and anything else would interface to it for Cocoa native access. It also supported callback command execution into the HalfPipe shell so HalfPipe could be scripted in JRuby.

Rhino JavaScript Script Engine (1.0.0) Before openjdk shipped it's Rhino engine I added this one of my own. That allowed me for one thing to add some code to JMX pid attach to any local java task. I modified a lot of the openjdk provided sample javascript to do this for JConsole to my app where, again, it could be used on any OS X java task with a pid. It also supports the same sort of callback back into the HalfPipe shell. So HalfPipe could be scripted in javascript.

REngine (0.7) And now R. With a current interest in machine learning this one is almost a must. I also added the jar for Weka (latest 3.7) to classpath. Now I just have to something with them.

From an earlier update...
Am I learning anything from all of this? Well I just retried the Kaggle Titanic competition using some of the R and Weka I've learned. A rough draft description of that here. Judge for yourself.

The Weka Kaggle Titanic info being updated should be a to-do list item. I had some better success although still have been unable to crack the 80% accuracy mark. I may yet make a few more attempts with ensemble stacking, as detailed here Weka Tutorial 13: Stacking Multiple Classifiers I also added a couple more arff files with competition related data that could be used by those trying out Weka with the Titanic competition. Data changes alone did not seem to get me significant improvements.

01/22/14 It occurred to me that a little information concerning Weka on OS X might be useful to someone. I put that here

I am trying to find time to back off some of the education related and do a little tinkering again. Besides the R interface added to the HalfPipe application mentioned below, items currently sort of on the to-do list would include...


I did have one idea to start applying some of what I am learning to the HalfPipe application. First, add a JSR 223 implementation of the R language. I found one, but haven't got it working yet. Then, use that in conjunction with Weka for a REPL machine learning/data mining interface.

Not REPL in the sense that HalfPipe is REPL. Yes, it reads, evaluates, prints if needed and loops. But, normally it provides little in the way of 'scope'. When I implemented Rhino and JRuby for HalfPipe I added two parts for each. For each, a JSR 223 command was included that reads a script either from System.in command line or from a indicated file and runs it. The second part was where a'scope'd environment figured in. For Rhino I also included the release provided javascript shell. For JRuby I found source for the java implementation of the IRB shell. These provide a command line scope where you can assign things and then manipulate them, etc. This is an extra dimension I haven't figured out how to do with the JSR 223 implementations yet. This could be my incomplete understanding of that api. However, it is a dimension that I feel is very important to an implementation of a command line language or scripting api.

Anyhow, the idea is to put together a scoped environment for data mining. Use weka for the machine learning parts, R for data manipulation and maybe data representation, whatever else for whatever else. But, interactive and scoped.

Thats the theory anyhow, so far in the time I've put into it I haven't even gotten the 'R' JSR 223 interface working.

I would, of course, also still like to figure out my trz kqueue deadlock when there are lots of events, and have one or two other things I'd like to put time into as well. If and when I find that time.

Also note, I did my own JSR 223 interface for Rhino before openjdk on OS X was including any javascript JSR 223 interface at all. I added additional functionality to the ScriptableObject to include a couple of the useful Rhino shell functions, also a function to JMX pid attach to any other java application, and finally a function to do callback HalfPipe itself shell commands. This gave me functionality I don't think Nashorn or any other shipping JSR 223 interface would give me. Looking at Nashorn now that it is included with JDK releases remains on my to-do list.

The JMX pid attach allowed any java application to do JMX based javascript scripting of any other java application on the local machine. This was an extension of the jdk demo javascript scripting of the JConsole application, I think it was. I modified and extended these demo scripts for my own use.


Election Games 2012


The election is over

Not long between the program available and the election being over. It went more or less as expected. The Swing state simulation favoring Obama looked like it will probably be off four states. Assuming Obama gets Fl>orida which may never be decided. The simulation gave New Hampshire, Nevada, Colorado and Virginia to Romney and these all went to Obama. Going into this simulation and giving Obama the "Women's rights" scenario and then "Deficit" gives a result 100% matching the election outcome. You have to add "Deficit" to take away North Carolina which I think he was competitive in but didn't carry. I have replaced the pre-election Swing simulation with this, assuming Fl., accurate simulation of the election called "Swing Sweep - exact match".

It is my current plan to definitely come out with a improved post-election version of the application. Thanks for the interest.


The 2012 version is available.

On a limited basis ony at this point in time. The OS X dmg includes the new version as a Apple Java 6 JVM application. I planned on going with Java Web Start for all platforms but haven't got that deploying the new databases. A lot of the data is now in Java Derby SQL database tables, there will probably be a separate download for that at some point as the data could be useful standalone outside of the application.


Links A couple directly related to this program as it is for the 2012 version
David Leip's Atlas of U.S. Presidential Elections 2008 version used data from here, 2012 may use current for the same eventually
Vote Ratings 2011 - NationalJournal.com

Related Download

Includes some related use of the National Journal data. Demonstrates how the data could be used separately from the program.
Hotelling.dmg

Inspired by the DNC convention I decided to blog Whatever


This is the Presidential election simulation software that the media and the politicians don't want you to know about. Results Image


Results Image The only interactive memorabilia for the presidential election.

Prove that you're the pundit of the office, the clairvoyant of the classroom.

Be the first one on your block to correctly pick the presidential election results of all fifty states plus the District of Columbia.

You decide what issues matter
You decide who they matter for and how much
You decide the states where they matter
You decide the next President of the United States


Election Related Links

This is shareware ($10.95), see it before you even try it, and try it before you buy it. See a applet version

Download the Apple Java 6 JVM OS X application .dmg
Download 7 the openjdk Java 7 JVM application .dmg


Java Web Start The Java Web Start launch
Java Web Start doesn't appear to work Safari on OS X but does with FireFox and so far IE on Wndows XP

EMail: contact



Sunday Sudoku Solver


When the puzzles get tough, the tough get the Sunday Sudoku Solver

SSS Image


Results Image Full featured software that lets you solve the toughest Sunday newspaper Sudoku puzzles.

This is shareware ($10.95), see it before you even try it, and try it before you buy it. See a
applet version

Download the OS X application .dmg

Launch the Java Web Start version. For Windows, other platforms, or if preferred on OS X.

EMail: contact


Official PayPal Seal


Update 08/17/14

Put the source and some old started documentation, I think already provided, on github

Source on GitHub

https://github.com/mik3hall/bjb

Back into this some starting 08/12/14.

Blackjack Blastoff Incomplete and buggy. Blackjack program with card counting. Various strategies, hi-lo of course, card counting tips and facts, charts. Play or run simulations. All that eventually. For what is there so far there is a Java Web Start version you could try...

Most of time still going into this. The program has progressed but has a ways to go. Java Web Start version continuously updated. Also starting work on having a couple independent documentation resources available, one with general information on blackjack and card counting. One, more an actual Blackjack Blastoff user guide. A link to the very brief start at the overview doc is below.

Java Web Start (jnlp)
Blackjack (html)
User Guide (pdf)

Earth Image


Tornado Alley

This map plots all tornado occurrences since the 1950's. It is based on data from Tornado History Project.

More on the this here for now. Another in progress thing.



The Turing Machine applet based on the book The Emperor's New Mind that I mentioned in the What's new section above. Basically, you select a program from the programs menu, then enter input and 'Exec' it or 'Step' through it. The UN+1 and UNx2 programs take one number input. The EUC, Euclid's algorithm program requires two numbers. So you sort of have an emulator for the Turing 1 computer. Complete with it's own language and compiler. Not bad for an applet? For more detail on the applet see Turing Machine


Java 7 on OS X


Update 08/17/14

Still not that much progress but pushed the current to github

I seem stuck on the LotsOfEvents test. It is seeming like it might be time to try a somewhat different approach. The current thought is do something based off of VDKQueue supposedly a newer version of the UKKQueue code that I used before. I am thinking I will leave it using notifications this time. I can come up with code to handle those from java like my HalfPipe application handles launch/termination notifications. Maybe this will avoid threading concerns or have other processing efficiencies that will allow it to handle LotsOfEvents better.

Otherwise this time I'm hoping to leave the native code as untouched as possible. One problem there though might be it not handling nested directories. I think I added code to the UKKqueue implementation for this. Since this is supposed to be a better and simplified version of that it probably doesn't include it either.

I did have one other thought for a simple WatchService implementation. Have the native code simply detect if a file system change of interest has occurred and then tell java. Java wakes up the OS X default Polling watch service which processes it immediately. This assumes that watch service can be woke up. Another change it would be nice to implement with that would be slow down as much as possible the normal polling to reduce overhead for that. Rely instead on the native 'wakeup'. It seems like it could be simple and reasonably effective, 'real time', until a better working more full-blown watch service is available.

I also came across SCEvents which is an FSEvents implementation off of Stuart Connolly's implementation which I think indicated was somewhat modeled again off of UKKQueue. But first something working for KQueue would be nice.

Putting more time into this 08/12/14/. Not much new progress yet.

Mac nio.2 platform specific file API extenstions.

Description: Uses JNI and the Java 7 nio.2 API's to provide a kqueue based WatchService and access to Mac OS X filesystem attributes and meta information.
Current OS X supported meta attributes for Finder, Launch Services, NSFileManager, and xattr
The current version is a beta for the kqueue based OS X WatchServic e.

UPDATE: 07.20.13

I have put a little more time into this lately. First trying to get the kqueue WatchService to work with a version of the jdk LotsOfEvents test. It tends to hang and stop posting events to the native queue when the java code starts waiting for more events. It also adds in support for the Apple eio FileManager class. So there is open source support for that if anyone wants its. I added a companion FileHelper class that...
1) Should make accessing trz meta attributes easier if you aren't familiar with the nio.2 api's yet.
2) Adds a couple methods to take advantage of the LaunchServices api's if you want to handle files with some other application.
These api's are largely untested and still need junit tests written. The
javadoc has been updated to reflect these changes and slightly improved itself. It probably could still be better.

This is also the first project I have attempted to move to GitHub. URL below.

Documentation

Javadoc

Project on GitHub

https://github.com/mik3hall/trz

Project update - 06.09.13

I was thinking that these java projects might be better as GitHub opensource ones. Putting them there would also give me a little experience working with and learning git. Having started this with trz I have decided to put a little more time into this particular project. Current to-do list things I might look at now would include...

LotsOfEvents Get the OS X kqueue nio.2 WatchService implementation to work with the jdk LotsOfEvents test. Looking at this it seems like it shouldn't really be all that much effort. The kqueue native code runs off it's own thread so should be able to keep posting out events. The java code would need to be modified to simply queue these into a collection when posted. Then wakeup up the WatchService poller thread to pull from and process this collection. This would be more consistent with existing WatchService implementations. This much I would definitely like to get done this pass through.

This would be a typical command line invocation of this test if I don't have one anywhere else...
/usr/libexec/java_home -v 1.7 --exec java -cp lib/macnio2.jar -Djava.nio.file.spi.DefaultFileSystemProvider=us.hall.trz.osx.MacFileSystemProvider LotsOfEvents

FSEvents Do a implementation of a WatchService based on this. It could be better suited to lots of files and my understanding is that it could capture events even when the java code isn't running. I would like to get into this but am unsure of the effort in getting it to conform to the java api's.

Dropbox I would like to get the custom filesystem providers into new stuff besides the usual reading, writing and arithmetic. Like allow for syncing with Dropbox. This is a 'I would kind of like to get into'.

Journaling Maybe try to support something for this. Maybe native, maybe not. Another 'might like to look at'.

Data policy To determine whether you sync or journal a file you might define a data policy. Maybe based off a xml or json config file. You could possibly make the data policy meta data for the file using OS X extended attributes. These are already supported read only as nio.2 attributes. This might be a reason to add application specific extended attribute update.

Any thoughts on what would interest you in a custom filesystem provider feel free too email me to express such interest.
EMail: contact

If you find the software useful or to support continued development consider a donation.


Half-Pipe - A platform for all kinds of coding and skateboarding tricks.

Update 2/20/14 Fix for launch hangs due to openjdk config bug. If the app didn't work for you at Mountain Lion or later, please try it again.

Description: A java command line shell that is my first attempt at a openjdk Java 1.7 application.
It is hoever, an update of a long supported Apple JVM Java application of mine. A full featured java shell with a variety of additional scripting support options.

The shell supports...
command piping, and redirection.
System properties as environment variables
Extended aliasing
Scripted automation (JMX/Rhino)

Scripting languages supported...
JRubyRhino
AppleScriptF-Script
and of course java

All that and a commandline too.

halfpipe7.dmg

Project Info

If you find the software useful or to support continued development consider a donation.


AppConverter - Convert Apple java applications to openjdk

Last updated 12/11/12

Description: Drag your Apple java applications to the AppConverter application and it will generate the appbundler build.xml file and create the openjdk application.

This should handle most if not all of the necessary work to convert an Apple Java application to an openjdk one. If your application is complicated some manual changes may be required after creating the openjdk one. If you let me know about any of those I can determine if they can be handled more automatically as well.

EMail: contact

AppConverter.dmg

Project info

If you find the software useful or to support continued development consider a donation.


Weka on OS X

You can obtain an OS X application bundle from the sourceforge project
Weka---Machine Learning Software in Java
Selecting the download should give you a dmg containing an application bundle for the Oracle JVM. You can simply drag that into your Applications folder to install. This application bundle does not include a embedded JRE. That means you need the 'common', applet, jre installed separately [I think this download Java SE Runtime Environment 7 Downloads]. I also copied the Weka-3-6-10 support folder to the Applications folder. Somewhere else might make more sense.

To include the optional, 3rd party, LibSVM classifier you need to download it from LIBSVM -- A Library for Support Vector Machines. Copy the libsvm.jar file from the Java folder to the Weka application bundle. The path would be the weka-3-6-10-oracle-jvm.app/Contents/Java. Control+click and select "Show Package Contents" to navigate to it. For Oracle AppBundler derived applications all jar files in this location will automatically be in classpath so you're done. You may have to move the bundle out of the Application folder before copying in the libsvm jar if file permissions don't allow update in the Applications folder. I don't remember off-hand, but that shouldn't be too difficult. Then just move it back.

Additional Weka reference


Miscellaneous

If of interest my personal home page, including some coding is...
home

I haven't been putting much time into shareware. So far the upcoming presidential election just doesn't interest me as much as the last one. The red shift in the midterm elections was sort of interesting and I was even considering attempting an update to ElectionGames then. But not at the moment.


A little coding project around oil and gas prices
gas.html