Just read this post...
Just saying... it's interesting. Seems to be a lot of the benchmarking bug going around these days.
Showing posts with label python. Show all posts
Showing posts with label python. Show all posts
Thursday, July 17, 2008
Wednesday, January 9, 2008
JRuby / Ruby Continuous Integration with Hudson
Hudson is a great tool for your continuous integration needs. It's very easy to setup and can run/monitor multiple types of tasks. Some have used it for Python and this blog entry will show you how to get Hudson setup quickly for running your Ruby or JRuby tests. (including reporting)
(This step-by-step is assuming you have Java installed on the machine you are using.)
First we'll need to download Hudson here and start our Hudson instance: (just grab the latest stable release)
[sourcecode language='css']
java -jar hudson.war
[/sourcecode]
(see Hudson site for other deployment options.)
Next, grab the CI::Reporter gem we'll need for test results xml formatting :
[sourcecode language='css']
gem install -y ci_reporter
OR for JRuby
jruby -S gem install -y ci_reporter
[/sourcecode]
UPDATE: Make sure you insert the following lines in your Rakefile:
[sourcecode language='ruby']
require rubygems
gem ci_reporter
require ci/reporter/rake/test_unit
[/sourcecode]
Hit your running Hudson instance: http://localhost:8080/ and configure a new build by clicking the "New Job" link. Enter the name of your build and select the "Build a free-style software project" (click OK.)
Configure your SCM data, then scroll down to the "Build" section and select either "Execute Windows batch command" (windows) or "Execute shell" (*nix). This is where we'll call our 'rake test' task using the CI::Reporter gem and specifying a spot for the report XMLs to live:
[sourcecode language='css']
//rake ci:setup:testunit test CI_REPORTS=results
OR for JRuby
//jruby -S rake ci:setup:testunit test CI_REPORTS=results
[/sourcecode]
UPDATE: When using Subversion, it will check things out into a sub directory of your "workspace" (found ~<user running hudson>/.hudson/jobs/<your job name>/workspace) If you are checking out 'trunk' and don't specify a different name for the directory, your shell execution settings should look like this:
(be sure to know how your SCM works before applying the following)
[sourcecode language='ruby']
cd trunk
//rake ci:setup:testunit test CI_REPORTS=results
[/sourcecode]
This will setup the CI::Reporter and place the XMLs in the "results" directory for us.
We are almost done, now under the "Post-build Actions" section, check the "Publish JUnit test result report" box and enter the following in the "Test report XMLs" field:
[sourcecode language='css']
results/*.xml
[/sourcecode]
Now click Save and go test it out... Select your newly created job from the Dashboard.
Click the "Build Now" link... done. After a few builds you should see a graph on the job's main page that looks like this:
If you have a failure, you can drill down by clicking the "Test Result" link on an individual build's page.
Clicking on an individual failed test's name will show you the output from that failure
That's all it takes, pretty simple and works like a charm. Now you can go back and play with the other settings Hudson has, triggers, scheduling, job artifacts, notifications... you could even have a job that prepares a build by generating all your static content so your server doesn't have to take the initial hit after startup. (web projects)
If interested, there is a way to create plugins for Hudson.
(This step-by-step is assuming you have Java installed on the machine you are using.)
First we'll need to download Hudson here and start our Hudson instance: (just grab the latest stable release)
[sourcecode language='css']
java -jar hudson.war
[/sourcecode]
(see Hudson site for other deployment options.)
Next, grab the CI::Reporter gem we'll need for test results xml formatting :
[sourcecode language='css']
gem install -y ci_reporter
OR for JRuby
jruby -S gem install -y ci_reporter
[/sourcecode]
UPDATE: Make sure you insert the following lines in your Rakefile:
[sourcecode language='ruby']
require rubygems
gem ci_reporter
require ci/reporter/rake/test_unit
[/sourcecode]
Hit your running Hudson instance: http://localhost:8080/ and configure a new build by clicking the "New Job" link. Enter the name of your build and select the "Build a free-style software project" (click OK.)
Configure your SCM data, then scroll down to the "Build" section and select either "Execute Windows batch command" (windows) or "Execute shell" (*nix). This is where we'll call our 'rake test' task using the CI::Reporter gem and specifying a spot for the report XMLs to live:
[sourcecode language='css']
/
OR for JRuby
/
[/sourcecode]
UPDATE: When using Subversion, it will check things out into a sub directory of your "workspace" (found ~<user running hudson>/.hudson/jobs/<your job name>/workspace) If you are checking out 'trunk' and don't specify a different name for the directory, your shell execution settings should look like this:
(be sure to know how your SCM works before applying the following)
[sourcecode language='ruby']
cd trunk
/
[/sourcecode]
This will setup the CI::Reporter and place the XMLs in the "results" directory for us.
We are almost done, now under the "Post-build Actions" section, check the "Publish JUnit test result report" box and enter the following in the "Test report XMLs" field:
[sourcecode language='css']
results/*.xml
[/sourcecode]
Now click Save and go test it out... Select your newly created job from the Dashboard.
Click the "Build Now" link... done. After a few builds you should see a graph on the job's main page that looks like this:
If you have a failure, you can drill down by clicking the "Test Result" link on an individual build's page.
Clicking on an individual failed test's name will show you the output from that failure
That's all it takes, pretty simple and works like a charm. Now you can go back and play with the other settings Hudson has, triggers, scheduling, job artifacts, notifications... you could even have a job that prepares a build by generating all your static content so your server doesn't have to take the initial hit after startup. (web projects)
If interested, there is a way to create plugins for Hudson.
Thursday, December 20, 2007
Sign of the times?
I was in a large book store chain recently and as I always do, I strolled down the technology isle and found this very interesting:
Is this a sign of the times? The heading tells me there will be "Java" books here... but the first two rows are all Python, PHP, Ruby/Rails and a little TCL book in there for good measure. :) Dynlangs (scripting languages) are beginning to take the front seat for development these days... do the folks at this store know this? Or was this a simple stacking slip?
With projects like JRuby, Jython and the like gaining steam as viable alternatives to the Java programming language on the JVM... maybe the heading should just read "JVM" and this stacking order would work just fine!
Thoughts?
Is this a sign of the times? The heading tells me there will be "Java" books here... but the first two rows are all Python, PHP, Ruby/Rails and a little TCL book in there for good measure. :) Dynlangs (scripting languages) are beginning to take the front seat for development these days... do the folks at this store know this? Or was this a simple stacking slip?
With projects like JRuby, Jython and the like gaining steam as viable alternatives to the Java programming language on the JVM... maybe the heading should just read "JVM" and this stacking order would work just fine!
Thoughts?
Friday, November 30, 2007
Interesting Debate...
Martin Fowler brings up a good topic on his blog : "GroovyOrJRuby"
I like the way Martin ends this post... Ruby (and Rails) have brought the dynamic (we used to call them scripting) languages back to the forefront.
There is a need for different languages, to help the programming/software engineering world grow. Finding different ways to do things. The same way we have Ruby, Python, Perl, PHP... we should have JRuby, Jython, PHP/Quercus, Groovy.
Which language is the best for the JVM? I would not jump right into saying Java... it really depends on what you are doing.
I like the way Martin ends this post... Ruby (and Rails) have brought the dynamic (we used to call them scripting) languages back to the forefront.
There is a need for different languages, to help the programming/software engineering world grow. Finding different ways to do things. The same way we have Ruby, Python, Perl, PHP... we should have JRuby, Jython, PHP/Quercus, Groovy.
Which language is the best for the JVM? I would not jump right into saying Java... it really depends on what you are doing.
Sunday, November 25, 2007
Simple file IO in different dynamic languages
Was just messing around and thought I would look at the different ways to skin a cat... in this case, the cat being "read each line of a file and spit out it's contents". I'm using TCL, Python, PHP, Ruby, Groovy, Scala and Javascript (Rhino) for this example. Let's take a look at what I came up with:
[sourcecode language="css"]
set fp [open "test.txt" r]
while {[gets $fp line] != -1} {
puts stdout $line
}
close $fp[/sourcecode]
Here we need to open the file pointer and loop over the file using a procedure call to 'gets' which sets a local variable 'line' for us. ('gets' is wrapped in brackets so we can validate on it's return, the length of the string returned by gets proc call.) Finally we will need to close our file pointer.
[sourcecode language="python"]
f = open("test.txt")
try:
for line in f:
print line.strip()
finally:
f.close()[/sourcecode]
Similar to TCL, we are getting our file pointer (in this case an object), looping the file object and closing the pointer. Version 2.6 will support the 'with' keyword allowing you to wrap the file IO in a block (therefor closing the file pointer for you after executing the block.) Here's what it will look like:
[sourcecode language="python"]
from __future__ import with_statement # <-- only use this line for Python 2.5!
with open("test.txt") as f:
for line in f:
print line.strip()[/sourcecode]
[sourcecode language="php"]
foreach (file('test.txt') as $line) {
echo $line;
}[/sourcecode]
This code looks 'blockish', but we're actually calling a function 'file' which returns the file's contents as an array... the file pointer is created and closed within this function so we don't have to worry about closing it in this example.
[sourcecode language="ruby"]
# File.open('test.txt', 'r').each do |line| <-- BAD: open file pointer
File.foreach('test.txt', 'r') do |line| <-- CORRECTION
puts line
end[/sourcecode]
UPDATE: was leaving file pointer open... thanks for the correction Sebastian Hungerecker.
Now we are getting OO with blocks. We're creating a File object and calling that object's 'each' method which iterates over each line of the file and passes the contents to the supplied block.
[sourcecode language="java"]
new File("test.txt").eachLine { line ->
println line
}[/sourcecode]
Again, OO with blocks... very similar to Ruby.
[sourcecode language="css"]
import scala.io.Source
for {
(line) <- Source.fromFile("test.txt").getLines
} print(line)[/sourcecode]
Procedural type programming using a language that 'goes both ways.' Similar to the PHP code above, the file IO is wrapped up in the 'Source.fromFile' method call which returns a 'Source' object already populated with the file's contents. We then call the getLines method which returns an iterable collection.
[sourcecode language="javascript"]
lines = readFile("test.txt").split("\n");
lines.pop(); // <-- last item is empty... EOF?
for (i in lines) {
print(lines[i]);
}[/sourcecode]
Using a Rhino built-in 'readFile' to get the contents of the file, splitting on line breaks using OO method call, and finally looping the array and printing it's contents. You see something I had to do here to get rid of the empty item in 'lines'.
Update: I failed to mention, these times include JVM startup so it's not a true speed check for IO... rather from the command line experience (total execution time of one call.) This can paint a bad light on the JVM impls, but keep in mind the JVM is built for long running rather than one shot processes.
Of course I was curious... what kind of speed do these languages have for this basic IO? The test file contained only the following:
I don't have the latest and greatest for a few of these... but here are the versions and real time taken. (using 'time' on Mac OS X 10.4.10)
[sourcecode language="css"]
RUBY: 0m0.009s (1.8.6)
TCL: 0m0.012s (8.5)
PHP: 0m0.024s (4.4.7)
Python: 0m0.034s (2.5.1)[/sourcecode]
[sourcecode language="css"]
RHINO: 0m0.310s (1.6R2)
GROOVY: 0m0.885s (1.0)
SCALA: 0m1.312s (2.6.0)
JRUBY: 0m1.668s (1.1b1)[/sourcecode]What does all of this mean? Not much really... just that there are a lot of different ways (many of which I didn't cover here) to skin a cat. Just choose the one that fits your needs and have fun with it.
Did I miss your favorite language here? Did I not use the most optimal code for accomplishing this task? Let me know!
Code
TCL
[sourcecode language="css"]
set fp [open "test.txt" r]
while {[gets $fp line] != -1} {
puts stdout $line
}
close $fp[/sourcecode]
Here we need to open the file pointer and loop over the file using a procedure call to 'gets' which sets a local variable 'line' for us. ('gets' is wrapped in brackets so we can validate on it's return, the length of the string returned by gets proc call.) Finally we will need to close our file pointer.
Python
[sourcecode language="python"]
f = open("test.txt")
try:
for line in f:
print line.strip()
finally:
f.close()[/sourcecode]
Similar to TCL, we are getting our file pointer (in this case an object), looping the file object and closing the pointer. Version 2.6 will support the 'with' keyword allowing you to wrap the file IO in a block (therefor closing the file pointer for you after executing the block.) Here's what it will look like:
[sourcecode language="python"]
from __future__ import with_statement # <-- only use this line for Python 2.5!
with open("test.txt") as f:
for line in f:
print line.strip()[/sourcecode]
PHP
[sourcecode language="php"]
foreach (file('test.txt') as $line) {
echo $line;
}[/sourcecode]
This code looks 'blockish', but we're actually calling a function 'file' which returns the file's contents as an array... the file pointer is created and closed within this function so we don't have to worry about closing it in this example.
Ruby
[sourcecode language="ruby"]
# File.open('test.txt', 'r').each do |line| <-- BAD: open file pointer
File.foreach('test.txt', 'r') do |line| <-- CORRECTION
puts line
end[/sourcecode]
UPDATE: was leaving file pointer open... thanks for the correction Sebastian Hungerecker.
Now we are getting OO with blocks. We're creating a File object and calling that object's 'each' method which iterates over each line of the file and passes the contents to the supplied block.
Groovy
[sourcecode language="java"]
new File("test.txt").eachLine { line ->
println line
}[/sourcecode]
Again, OO with blocks... very similar to Ruby.
Scala
[sourcecode language="css"]
import scala.io.Source
for {
(line) <- Source.fromFile("test.txt").getLines
} print(line)[/sourcecode]
Procedural type programming using a language that 'goes both ways.' Similar to the PHP code above, the file IO is wrapped up in the 'Source.fromFile' method call which returns a 'Source' object already populated with the file's contents. We then call the getLines method which returns an iterable collection.
Javascript (Mozilla Rhino)
[sourcecode language="javascript"]
lines = readFile("test.txt").split("\n");
lines.pop(); // <-- last item is empty... EOF?
for (i in lines) {
print(lines[i]);
}[/sourcecode]
Using a Rhino built-in 'readFile' to get the contents of the file, splitting on line breaks using OO method call, and finally looping the array and printing it's contents. You see something I had to do here to get rid of the empty item in 'lines'.
Speed?
Update: I failed to mention, these times include JVM startup so it's not a true speed check for IO... rather from the command line experience (total execution time of one call.) This can paint a bad light on the JVM impls, but keep in mind the JVM is built for long running rather than one shot processes.
Of course I was curious... what kind of speed do these languages have for this basic IO? The test file contained only the following:
this
is
a
test
I don't have the latest and greatest for a few of these... but here are the versions and real time taken. (using 'time' on Mac OS X 10.4.10)
Native Impls
[sourcecode language="css"]
RUBY: 0m0.009s (1.8.6)
TCL: 0m0.012s (8.5)
PHP: 0m0.024s (4.4.7)
Python: 0m0.034s (2.5.1)[/sourcecode]
JVM Impls
[sourcecode language="css"]
RHINO: 0m0.310s (1.6R2)
GROOVY: 0m0.885s (1.0)
SCALA: 0m1.312s (2.6.0)
JRUBY: 0m1.668s (1.1b1)[/sourcecode]What does all of this mean? Not much really... just that there are a lot of different ways (many of which I didn't cover here) to skin a cat. Just choose the one that fits your needs and have fun with it.
Did I miss your favorite language here? Did I not use the most optimal code for accomplishing this task? Let me know!
Tuesday, October 30, 2007
“DynLangs” and you…
Unless you have been living behind your monitor and keyboard only to look up to find the coffee mug you knocked over, you've most likely heard about all the attention dynamic languages, or dynlangs, have been getting these days. Dynamic Languages (or scripting languages) have been taking their spot in the limelight in recent years... I feel this is in large part due to the readily available video streaming sites showcasing the benefits of such languages/frameworks.
For those willing to step outside the compiler, large gains in productivity abound. And with so many dynamic language options being implemented on the JVM, JSR 223, and attention from large companies... dynlangs are quickly becoming viable options in companies and organizations that would never have considered using them in the past.
If you primarily work with a compiled language (Java maybe?) and don't want to miss the productivity boat, you may want to review the following list comprised of just a few languages/frameworks making waves (on and off the JVM) :
Languages
Ruby
Object Oriented, interpreted, dynamically typed language. Created by Yukihiro “matz” Matsumoto, everything is an object and every class is "open" for change.
JRuby
JVM implementation of the Ruby language... provides the ability to use the Ruby language and leverage the existing capabilities of the Java Virtual Machine (and all it's libraries.)
Groovy
Dynamic OO language built for the JVM... "builds upon the strengths of Java but has additional power features inspired by languages like Python, Ruby and Smalltalk"
Python
Dynamic OO language, embeddable, integration friendly, dynamically typed. Strong user base.
Jython
JVM implementation of the Python language... project has fallen behind the Python pace, but new life buds. As with most JVM dynlangs, Jython provides Java integration.
Scala
Type-safe, both object oriented and functional. Written for the JVM so the Java integration is there.
Rhino
JVM implementation of the JavaScript language... all you would by now expect from a JVM dynlang, Java integration, embeddable, shell access.
... and of course...
PHP
"PHP is a widely-used general-purpose scripting language that is especially suited for Web development and can be embedded into HTML." Heavily pervasive, low barrier to entry, object oriented with PHP 5, large library code base.
(PHP can now run on the JVM too: Quercus)
Web Frameworks
Ruby on Rails
"Rails is a full-stack framework for developing database-backed web applications according to the Model-View-Control pattern. From the Ajax in the view, to the request and response in the controller, to the domain model wrapping the database, Rails gives you a pure-Ruby development environment. To go live, all you need to add is a database and a web server." ORM, MVC, JavaScript and templating for views...
Grails
"It's an open-source web application framework that leverages the Groovy language and complements Java Web development." Inspired by the Ruby on Rails framework (at least at one time it was.)
django
"Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design." ORM, MVC we're seeing a pattern here...
Symfony
"Based on the best practices of web development, thoroughly tried on several active websites, symfony aims to speed up the creation and maintenance of web applications, and to replace the repetitive coding tasks by power, control and pleasure." (PHP web framework)
Did I miss a dynlang/framework you enjoy working with? (most certainly...) Then drop me a line in the comments below and let me know what dynamic language or framework you use to get your job done!
P.S.
One of the best implementations of a dynamic/scripting language IDE around (non-trivial task): NetBeans Ruby Support (thanks Tor)
Subscribe to:
Posts (Atom)