Discussion:
Ants: Clojure vs. Common Lisp
(too old to reply)
Xah Lee
2009-06-15 21:15:01 UTC
Permalink
Hi,
I have reimplemented the Ants simulation example that comes with Clojure
in Common Lisp, more precisely in LispWorks, using its multiprocessing
and graphics capabilities.
No, you haven't. What you've implemented is a pale imitation of the
ants simulation that neglects its most important features.
Furthermore, it doesn't use STM as in Clojure, but rather a solution
based on locks, which doesn't seem to be much harder to me and required
only very little tweaking. Since there is no need to monitor read and
write accesses to shared mutable state in the lock-based version, it
should also be more efficient.
It doesn't come close to doing the same job.
The ants simulation was designed to show how to do things that are
difficult to do without an STM. It is not simply a demo of how to use
multiple threads to make ants move around on screen.
STM lets you make decisions based upon consistent information. So,
e.g. in the Clojure version when the ant looks around and sees food on
an empty space, and makes a decision to pick it up, it will be able to
do so without finding it missing or the space occupied, i.e. it will
atomically be able to make a decision and act upon it, even one
involving multiple ad hoc elements. In your code you've made it so the
ants can't see each other, thus removing the coordination problem
rather than solving it. The Clojure code can handle arbitrary sets of
read/modify operations, on arbitrary sets of involved data,
consistently.
You use timeouts on your locks, and silently fail to do the requested
operation if the lock times out. But the caller of move expects it to
work - it's not called maybe-move-if-we-can-get-the-lock. What if
evaporate did something important? It wouldn't be ok for it to skip
some steps if it can't get the locks.
There's more. You hold locks on an ongoing basis, vs obtaining and
releasing in a scope - always tricky and dangerous. Do you know how
that will behave should those locks map to OS resources? How might non-
scoped locks thwart optimization? In move, you obtain the new lock, do
some work, then unlock the old, and right now that work might be
trivial. But what happens when someone comes along later and makes the
work something that can fail? You risk orphaning a lock on such
failure. You simply cannot use locks without proper error handling.
The Clojure ants sim also tackled the 'reporting' problem. How can you
get a consistent view of a world with many concurrent updates without
stopping it? The Clojure render function works with a consistent
snapshot of the world. There's no chance of it drawing the same ant
twice, or two ants in the same place. You've taken all the challenge
out of it by removing correlated data. Still, your code can show a
view of the world that never existed. You might think it's no big deal
when you are just painting in a game-like sim, but the techniques in
the Clojure version work in serious situations when related things
need to add up, and yours simply won't.
In short, your version doesn't compare whatsoever to what the Clojure
version provides. By making the problem simpler you've destroyed its
purpose, and you've avoided the illustrative challenges.
The purpose of the ants sim was to demonstrate techniques for handling
multithreaded applications where units of work involve consistent,
arbitrary, overlapping subsets of information, and, reporting
consistently on correlated data subject to concurrent modifications.
The Clojure version demonstrates that and your version does not.
Rich
Great info. Thanks.

PS i'm spreading this piece of writing to comp.lang.scheme and
comp.lang.functional. I'm doing this on my own accord, because i
believe it is beneficial in several aspects.

Xah
∑ http://xahlee.org/


Jon Harrop
2009-06-16 04:07:51 UTC
Permalink
Post by Xah Lee
PS i'm spreading this piece of writing to comp.lang.scheme and
comp.lang.functional. I'm doing this on my own accord, because i
believe it is beneficial in several aspects.
Very interesting, thanks. This sounds like a great STM example...
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
ok
2009-06-16 12:09:22 UTC
Permalink
[ begging sympathy ]
2910 (mostly) negative votes, only on this one of your (many many)
accounts??

Now you only need to reverse your polarity, and you will finally be
some Holy Master of Universe!
Benjamin L. Russell
2009-06-17 04:51:48 UTC
Permalink
Post by ok
[ begging sympathy ]
2910 (mostly) negative votes, only on this one of your (many many)
accounts??
Now you only need to reverse your polarity, and you will finally be
some Holy Master of Universe!
Um, assuming you are referring to his vote on Google Groups, that may
or may not be so, but how is this related to the topic of this
newsgroup? I thought we were here to discuss issues, not
personalities; it's appropriate to express whatever opinion we have,
but can't we keep the discussion from becoming personal? Let's attack
ideas, not people.

I've seen _non sequitur_ and _ad hominem,_ but rarely together in one
expression. :-(

-- Benjamin L. Russell
--
Benjamin L. Russell / DekuDekuplex at Yahoo dot com
http://dekudekuplex.wordpress.com/
Translator/Interpreter / Mobile: +011 81 80-3603-6725
"Furuike ya, kawazu tobikomu mizu no oto."
-- Matsuo Basho^
Jon Harrop
2009-06-17 21:10:34 UTC
Permalink
Post by Benjamin L. Russell
Post by ok
[ begging sympathy ]
2910 (mostly) negative votes, only on this one of your (many many)
accounts??
Now you only need to reverse your polarity, and you will finally be
some Holy Master of Universe!
Um, assuming you are referring to his vote on Google Groups, that may
or may not be so, but how is this related to the topic of this
newsgroup? I thought we were here to discuss issues, not
personalities; it's appropriate to express whatever opinion we have,
but can't we keep the discussion from becoming personal? Let's attack
ideas, not people.
I've seen _non sequitur_ and _ad hominem,_ but rarely together in one
expression. :-(
Given that everything I write on usenet gets a single downvote on Google
Groups, it is obviously a bot. He's probably the author of the bot. Or
maybe he *is* the bot. Either way, surely it is even sillier to engage him
(it?) in conversation! ;-)
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
fft1976
2009-06-18 18:15:00 UTC
Permalink
What bothers me a little bit about Clojure's overall philosophy is
that it uses OOP (in one of the many senses of the term) like the
"sequence abstraction", etc., but at the same time, it discourages OOP
(such as creating your own abstractions).

It feels like biting the hand that feeds you.
jgrant27
2009-06-18 20:45:23 UTC
Permalink
Post by fft1976
What bothers me a little bit about Clojure's overall philosophy is
that it uses OOP (in one of the many senses of the term) like the
"sequence abstraction", etc., but at the same time, it discourages OOP
(such as creating your own abstractions).
It feels like biting the hand that feeds you.
Yes the larger issues of Clojure's philosophy bother me to.

STM is cool but it comes with it's own issues. There's no silver
bullet for concurrent programming.
Pushing the use of STM along with the idealistic but impractical
purely "functional" use of data structures on programmers, as part of
Clojure's philosophy, is where I take issue. Let the programmer decide
what works best for their needs without making too many important
assumptions for them. That is the Java way.

There was a more detailed discussion on concurrent/multi-core
programming regarding "purely" functional languages (Clojure /
Haskell) vs. Common Lisp here --> http://clozure.com/pipermail/openmcl-devel/2009-May/009516.html
Rich Hickey
2009-06-18 21:19:11 UTC
Permalink
Post by jgrant27
Post by fft1976
What bothers me a little bit about Clojure's overall philosophy is
that it uses OOP (in one of the many senses of the term) like the
"sequence abstraction", etc., but at the same time, it discourages OOP
(such as creating your own abstractions).
It feels like biting the hand that feeds you.
Yes the larger issues of Clojure's philosophy bother me to.
STM is cool but it comes with it's own issues. There's no silver
bullet for concurrent programming.
Pushing the use of STM along with the idealistic but impractical
purely "functional" use of data structures on programmers, as part of
Clojure's philosophy, is where I take issue. Let the programmer decide
what works best for their needs without making too many important
assumptions for them. That is the Java way.
There was a more detailed discussion on concurrent/multi-core
programming regarding "purely" functional languages (Clojure /
Haskell) vs. Common Lisp here -->http://clozure.com/pipermail/openmcl-devel/2009-May/009516.html
A language comes with some data structures. Each is either immutable
or not (they can't be both!). Clojure's providing some immutable data
structures is no more it 'deciding for' or 'pushing' the programmer
than is Common Lisp's decision to provide mutable ones (and no,
choosing not to mutate a mutable data structure does not convey the
benefits of an immutable persistent data structure - you can't make a
cheap copy, you can't trust it won't be accidentally modified once
shared, the optimizer can't leverage the immutability etc).

If you want to program with mutable data structures, locks, etc with
Clojure you certainly can. In fact, if you want to use Clojure's
sequence library with mutable data structures you can. Can I use CL's
sequence library with immutable data structures? No - it is hardwired
to the (very few) mutable ones CL comes with. How is that letting the
programmer decide?

Rich
jgrant27
2009-06-18 23:18:53 UTC
Permalink
Post by Rich Hickey
A language comes with some data structures. Each is either immutable
or not (they can't be both!).
Haskell (a 'purely' functional language) has mutable and immutable
versions of the same data structures :
e.g. Data.Array.MArray vs. Data.Array.IArray
http://hackage.haskell.org/packages/archive/array/0.1.0.0/doc/html/Data-Array-MArray.html#7
Also, for immutable/mutable arrays in Haskell there are functions to
convert/copy between each kind.
See freeze, unsafeFReeze, thaw and unsafeThaw.
The GHC compiler also does the write kinds of optimizations('cheap
copies') between certain pairs of array types.
All this allows the programmer to make the decisions.
Post by Rich Hickey
(and no,
choosing not to mutate a mutable data structure does not convey the
benefits of an immutable persistent data structure - you can't make a
cheap copy, you can't trust it won't be accidentally modified once
shared, the optimizer can't leverage the immutability etc).
All this can be implemented in CL (but convention goes a long way
instead).
As for STM, there are libraries (and not part of the language) for
this --> http://common-lisp.net/project/cl-stm/doc/Examples.html#Concurrent_0020counter


-Justin
Scott Burson
2009-06-19 05:10:28 UTC
Permalink
Post by Rich Hickey
If you want to program with mutable data structures, locks, etc with
Clojure you certainly can. In fact, if you want to use Clojure's
sequence library with mutable data structures you can. Can I use CL's
sequence library with immutable data structures? No - it is hardwired
to the (very few) mutable ones CL comes with.
Not if you use FSet :)

(I know you know that -- I just couldn't resist the opportunity to
plug FSet :)

-- Scott
Scott Burson
2009-06-19 05:10:28 UTC
Permalink
Post by Rich Hickey
If you want to program with mutable data structures, locks, etc with
Clojure you certainly can. In fact, if you want to use Clojure's
sequence library with mutable data structures you can. Can I use CL's
sequence library with immutable data structures? No - it is hardwired
to the (very few) mutable ones CL comes with.
Not if you use FSet :)

(I know you know that -- I just couldn't resist the opportunity to
plug FSet :)

-- Scott

Paul Rubin
2009-06-19 00:45:21 UTC
Permalink
Post by jgrant27
Pushing the use of STM along with the idealistic but impractical
purely "functional" use of data structures on programmers, as part of
Clojure's philosophy, is where I take issue. Let the programmer decide
what works best for their needs without making too many important
assumptions for them. That is the Java way.
Replace the acronym "STM" with "GC", and "functional use" with
"automatic management", and we heard the exact same song about C
vs. Java.
jgrant27
2009-06-19 02:42:12 UTC
Permalink
Post by Paul Rubin
Replace the acronym "STM" with "GC", and "functional use" with
"automatic management", and we heard the exact same song about C
vs. Java.
C and Java are imperative languages. We're talking about functional
languages here, either 'pure' or 'impure'.
This is a critical difference. STM with imperative languages means
that we are dealing with mutable data structures while in functional
languages we may or may not be dealing with mutable data structures.

STM is a good tool in many situations where we want to perform
concurrent programming WITH MUTABLE TYPES. The whole point of using
STM is to read/write something safely and so immutable data structures
don't add much value here. Even in the Haskell 'purely functional'
camp which has been around a while, there are many valid reasons that
they have Monads, immutable/mutable data structures and ways to modify
them. Things are usually immutable in Haskell by default but that does
not mean state cannot be changed, just that when it's done it's
explicit.

Of course there are many other valid uses for immutable data
structures too but why are they required with STM in Clojure ? If we
use immutable data structures in Clojure then why do we need STM at
all ?
In Clojure they are probably required because there are layers of type
systems. The Clojure types(immutable) live on top of Java types
(mutable). Then there is also the Java interop ability of Clojure
which means that STM does have value because we could be dealing with
mutable Java types. This is a bizarre mix to have to keep in mind when
trying to write software (let alone concurrent software).

If STM is being used for concurrent programming then the argument for
immutable types only seems to lose weight more quickly in Clojure(or
any other language). Optimization hints, 'cheap copies' (just
pointers ?) and ensuring no accidental changes with mutable data
structures is easy enough. C has it's 'const', Java has it's 'final'
keyword, Lisp has defconstant etc etc.


-Justin
Steve Schafer
2009-06-19 03:42:40 UTC
Permalink
Post by jgrant27
Post by Paul Rubin
Replace the acronym "STM" with "GC", and "functional use" with
"automatic management", and we heard the exact same song about C
vs. Java.
C and Java are imperative languages. We're talking about functional
languages here, either 'pure' or 'impure'.
This is a critical difference. STM with imperative languages means
that we are dealing with mutable data structures while in functional
languages we may or may not be dealing with mutable data structures.
I think you're kind of missing Paul's point...
Post by jgrant27
...
Of course there are many other valid uses for immutable data
structures too but why are they required with STM in Clojure ? If we
use immutable data structures in Clojure then why do we need STM at
all ?
And, apparently, the point of STM in Clojure, too. STM, much like IO and
other monads in Haskell, is a way of introducing mutability in a
_controlled_ fashion, so that you can get your work done without having
to worry about all of the nasty issues that would otherwise arise in
concurrent programming.

I think you're off the mark regarding Clojure's philosophy, too. If I
might paraphrase (and anthropomorphize): "Hi, my name is Clojure. You
know something? Functional programming with higher-order functions is
great stuff. Your code is much simpler and more concise, and is more
likely to work the way you want it to. Yeah, I know, some kinds of
things are just plain hard to express in pure functional code. But don't
worry, I've got some cool stuff for that, too. One of the neatest things
is called STM, and it makes stateful, concurrent programming a whole lot
easier."

Steve Schafer
Fenestra Technologies Corp.
http://www.fenestra.com/
Jon Harrop
2009-06-19 08:27:54 UTC
Permalink
Let the programmer decide what works best for their needs without making
too many important assumptions for them. That is the Java way.
Nonsense. Java is the OO+generics way or the high way.

If Java really did leave it up to the programmer to make the right decision
then it would be a multiparadigm common-language platform with first-class
lexical closures, value types, tail calls, a fast FFI, a decent static type
system etc.

In reality, Java seriously penalizes the programmer for even attempting to
do anything slightly outside its very limited design constraints. Even
generics were half-assed.

The CLR does a much better job of leaving it up to the programmer. If you're
writing C# and want to do logic programming or tree munging you can drop to
a more appropriate language like F# seamlessly. If you're writing F# and
want to write Fortran-style numerical code or use established
code-generating tools then you can drop to C# seamlessly. You can lash
together VB and interoperate with that seamlessly too. There are still
important design limitations but it is by far the most versatile
industrial-strength platform in existence today.
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
jgrant27
2009-06-19 08:37:43 UTC
Permalink
Post by Jon Harrop
Let the programmer decide what works best for their needs without making
too many important assumptions for them. That is the Java way.
Nonsense. Java is the OO+generics way or the high way.
You either misread this or I should have phrased it better, I actually
agree with you.
The Java way assumes that the Java language designer knows best and
will make many assumptions for the Java programmer.
I have always had a problem with this.
Jon Harrop
2009-06-16 04:07:51 UTC
Permalink
Post by Xah Lee
PS i'm spreading this piece of writing to comp.lang.scheme and
comp.lang.functional. I'm doing this on my own accord, because i
believe it is beneficial in several aspects.
Very interesting, thanks. This sounds like a great STM example...
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
Loading...