Artificial Intelligence, Neuroscience, Quantitative Finance and the unedited thoughts of a soon-to-be robot

Updates from May, 2009 Follow me on Twitter | View my gallery | Show Articles | Show Blog Posts

  • Rod Furlan 10:35 pm on May 30, 2009 Permalink | Reply  

    Computer Science Projects You Should Know About 

     

    Intel Larrabee GPGPU

    http://www.intel.com/technology/visual/microarch.htm
    http://en.wikipedia.org/wiki/Larrabee_(GPU)

    What is it?
    Larrabee is a hybrid CPU-GPU processor with an ingenious design that might greatly accelerate the adoption of massively parallel desktop computing.

    Why it is awesome:
    Larrabee is Intel’s attempt to bridge the gap between general purpose CPUs (Pentium, PowerPC, etc) and massively parallel GPUs (Quadro, GeForce, Radeon) by offering a hybrid chip that will (supposedly) deliver the flexibility and ease of development of common CPUs coupled with the massively parallel features of GPUs. The first Larrabee model is supposed to ship with 32 independent processing cores on Q2 2010 followed by a 48 core model on Q4 2010.

    Throughput vs Programmability – The Larrabee advantage

    Larrabee is Intel’s test-bed for several many-core computing technologies. If successful it will greatly accelerate the transition to many-core desktop computing – imagine having 128, 256 or 512 general purpose processing cores on your desktop.

    Possible outcome:
    Supercomputer-grade power, on your desktop.


     

    Numenta

    http://www.numenta.com

    What is it?
    “Numenta is creating a new type of computing technology modeled on the structure and operation of the neocortex. The technology is called Hierarchical Temporal Memory, or HTM, and is applicable to a broad class of problems from machine vision, to fraud detection, to semantic analysis of text.”

    Why it is awesome:
    Numenta’s platform for Intelligent Computing is based on the theory of the neocortex described in Jeff Hawkins‘ book “On Intelligence“. If Jeff’s assumptions regarding the inner workings of the neocortex are correct, HTM networks could give rise to the first generation of truly intelligent machines.

    Possible outcome:
    Artificial General Intelligence.


     

    Evolved Machines

    http://www.evolvedmachines.com

    What is it?
    “We simulate neuronal components closely modeled after neurons in the brain, and synthesize arrays which wire themselves by simulating neural circuit growth in 3-dimensions. We are the first company to harness the power of programmable GPUs to the simulation of neural computation, now achieving 100-fold acceleration of the computing power of conventional cores”

    Why it is awesome:
    Never mind the fact that they are “growing neuronal circuits” and that they have made great progress towards the creation of an artificial visual cortex. What makes their technology awesome is the fact that it was designed to run on massively parallel GPUs instead of conventional general purpose CPUs.

    Puny CPU vs. Mighty GPU:

    To put it in perspective, the first programmable GPU made available in 2001 had a theoretical peak performance of 71 GFLOPs which is about as much as an Intel Core 2 Quad CPU delivers today (2009). A current consumer-grade (read as: cheap) GPU can deliver up to 1,788 GFLOPs.

    Once you have expressed an algorithm in a way that it can be executed on massively parallel hardware it then becomes trivial to run larger (and then larger) experiments. Top that with the fact that GPU performance is doubling roughly every 16 months and you have an exponential winner.

    Possible outcome:
    Artificial General Intelligence – or at least sniper kill-bots that never miss.


     

    Boinc

     

    http://boinc.berkeley.edu

    What is it?
    The Berkeley Open Infrastructure for Networked Computing (Boinc) is an open-source platform for volunteer computing and grid computing. It allows users to donate the idle time on their computer to cure diseases, study global warming, discover pulsars, and many other types of scientific research.

    Why it is awesome:
    It provides scientists with supercomputer-grade processing power at no cost. If this isn’t awesome, I don’t know what awesome is.

    Also, the total donated GFLOPs will (theoretically) grow at the same pace as the internet and will be further amplified by the growth of processing power available on desktop computers.

    Good looks don’t make supercomputers any faster:

    Possible outcome:
    Communication with aliens (SETI@Home), cure for HIV, Malaria, Cancer, and Alzheimer’s (Rosetta@Home), figuring out if adiabatic quantum computers can compute anything useful (AQUA@Home), etc.


     

    Blue Brain

    http://bluebrain.epfl.ch

    What is it?
    “The Blue Brain project is the first comprehensive attempt to reverse-engineer the mammalian brain, in order to understand brain function and dysfunction through detailed simulations.”

    Why it is awesome:
    While most attempts to create functional simulations of the human brain are based on highly stylized models of neuronal activity, the smart folks at the EPFL are instead pulling all stops and trying to emulate neurons down to the molecular level. Their approach involves first simulating a small cluster of cortical neurons (a column) and then validating the simulation against extensive observations from experiments with real cortical columns until the simulation can be considered bio-identical.

    Possible outcome:
    Artificial General Intelligence and final confirmation that we don’t have souls.

     
  • Rod Furlan 12:57 pm on May 25, 2009 Permalink | Reply  

    In Defense of Efficient Computing 

    We all know that the cost of computing is on a definitive downtrend and while this is a great thing I worry that it is also steering developers to become less proficient in writing efficient (and reliable) code.

    If most problems can be solved by throwing money and hardware at sub par wasteful code, what is the incentive to writing thoughtful, efficient programs?earth

    The answer is RELIABILITY (and possibly to save your planet)

    While the cost of computing is going down the adoption of dynamic languages is going up and that worries me. Perhaps I am just a backwards man who hasn’t seen the light yet but I urge you to bear with me as I present my argument against the use of dynamic languages in a production environment.

    Why do I claim that dynamic languages are wasteful?

    tip If you appreciate theater it is likely that you also appreciate that  the actors have to work hard at every performance. It is a beautiful thing, but it is also a lot of repetitive work and that is exactly the problem with dynamic languages in general.

    How so? Every execution (performance) requires that a lot of work is done by the language runtime (showtime) environment. Most of the time this is repetitive work, mindlessly repeated every time you run your code.

    600x400_gallery_SC_Movie_Theater_1 Compare this with the stark contrast of cinema (compiled code) – there is no showtime hustle, the movie is already shot, cut, edited and packaged for delivery. No energy is wasted.

    You might ask me how significant is this waste I am talking about and the truth is that most of the time it is negligible but like everything in life, the devil is in the details.

    Hypothetically, let us say that at each execution a dynamic program wastes a mere 10,000 CPU cycles doing “preventable runtime busy-work”. That figure is utterly insignificant if you are looking at the trees only, but as you step back to look at the forest your perspective might change.

    Imagine this same piece of code running on a few hundred-thousand computers around the world and being executed a few thousand times a day on each. Now multiply this by each program of each dynamic language in existence and you might conclude that dynamic languages are not very good for our planet because computing demands energy and energy demands natural resources.

    So, if you believe in green computing through algorithmic efficiency you already have a good case against the use of dynamic languages.

    “But dynamic languages are more productive so the energy wasted on one end is saved on another!”

    Some might argue that dynamic languages are able to offset their “global footprint” by being more productive. Unfortunately, I beg to differ.

    Unless you are simply writing throwaway (or generally short-lived) code, the odds are that whatever you are building will remain in play for years to come. This brings two new variables into the equation, one is maintenance costs and the other is failure costs. I am going to argue that compiled languages do a better job at minimizing those costs in the long run due to the increased reliability they offer.

    First, we need to define what “productive” means in a quantifiable way because there is no room for fuzzy subjective opinions in computer science.

    I believe that the Total Cost of Ownership (TCO) of a software solution is the ideal metric of productivity. A language that can deliver a piece of software with a lower lifetime TCO should be considered more productive simply because more was produced for less.

    TCO encompasses everything from time-to-market risk, development costs, maintenance costs, failure-rate costs as well as any other overhead incurred.

    My argument is that while dynamic languages may accelerate the initial delivery of a software solution, it does so at the cost of reliability and maintainability. This is detrimental for the TCO bottom line because in practical terms (at least for large, long-lived projects) it translates into decreased productivity.

    “But compiled languages are too hard!”

     

    The place for dynamic languages

    With all that being said, I must disclose that I often use dynamic languages in my research when it makes sense, mostly for throwaway code or prototyping. In fact, one of my favorite tools (MATLAB) is centered around a particularly terrible dynamic language. What I am against is using dynamic languages for the development of code intended for production environments. That is all.

    Conclusion

    If you love our planet and you love computer science, I urge you to not take the easy way out, embrace the beautiful complexity of machine efficient code and write greener programs by using a compiled language. It can only make you smarter in the long run :)

     
    • Ken Dakin 9:58 am on September 12, 2011 Permalink | Reply

      Rod,
      I completely agree with your article and always wonder where people are coming from when they happily waste resources just because they can get away with it. Having faster processors is good because we can do lots more work for the same cost (or get faster answers to time critical questions) – not because we can acheive the same result by allowing ourselves to get lazy. I agree that for one-off programs that are run a couple of times it doesn’t matter too much – but one-off programs have a tendency to outlive their creators and get modified to create more one-off programs that may also persist. This way lies madness (non-maintainable code that is both inefficient and inconsistent).
      For production programs we should construct solid frameworks that are as efficient as possible (and as re-useable as possible unless this conflicts with performance to a great extent). Most of the programs and sub-routines I produced over 40+ years were written with efficiency constantly in mind and yet they also had close to zero bugs and needed few changes if any over their very long lifetimes of 20 years or more. I produced instruction set simulators for test/debugging purposes where zero tolerance was the only option (otherwise the products would have been worse than useless and given false positives).
      I would like to say that I very much enjoy your blog and I am as impressed, as you clearly are, by the awe inspiring JFK speech which always gives me goosebumps each time I see it. My own wish is that world leaders get together soon and jointly commit to setting up a base on Mars by 2030 say. This might also kick start the world economy for years to come and inspire a new generation like nothing else could. My audacious vision is that we should send self assembly robots to carve out an underground base first using only materials to hand (e.g. to fashion most of the ‘boring components’ out of natural rock for instance – at ever increasing scales) and follow this with a manned expedition to secure and populate the prepared space.

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
esc
cancel