Andy's Blog

JavaOne: TS-5471: Jini and JavaSpaces Technologies on Wall Street

Jini is a tool for building SOA's. The Master/Worker pattern is a common one in Jini systems.

Generic Virtual Data Access Layer (SOA)

• Distributed data, bring together (left joins?)
• Data is federated (many masters)
• Expose common Grid-like API for clients (JAR), or WebServices, or even JDBC
• Needs virtual data dictionary, metadata to describe data, to glue together
• Decompose data access into generic workers -- break down request/response into many sub-tasks
• Decouple SQL query generation from execution
• Distribute workers, get parallelism

Scalable Real-time Transaction Processing

• Parallelize transaction w/o losing FIFO ordering, and still getting 100% reliability
• Clustered JavaSpaces -- make several instances. Make reliable by replicating data in these instances.
• Build smart proxy that handles JavaSpace client requests, and distributes work into JavaSpaces cluster
• Automate deployment and restart of JavaSpace instances, using dynamic service-grid architecture -- gets you dynamic scalablity (apache httpd forking style)

Why now?

• memory was expensive -- not anymore
• Bandwidth was a bottleneck -- not anymore (GigE, 10Gig)
• Commodity HW finally enterprise grade

Building a Distributed Compute Grid Framework

• design grid applications incrementally with Jini and JavaSpaces
• from programmers perspective:
  1. how design application?
  2. how implement the design?
• too much talk about design, not enough about programming
• Good design always starts with something simple and evolves -- Jini and JavaSpaces make this easy: loosely coupled components, dynamic and flexible infrastructure
• Features can be added as needed

Grid computing framework

• master-worker pattern based compute farm
• A layer of abstraction over JavaSpaces API and Jini programming model
• Framework class design
  • Decomposer - concrete class will decide what correct subtask size is
  • Distributor
  • Calculator (processor)
  • Collector
  • Task
  • Result
  • Communicator - communication and synchronization among compute nodes

Putting the Spring into Grid

• have to consider user interface for the programme
• How can we take POJO model and bring it to Jini/JavaSpaces?
• Get a lot of power in POJO approach, because we decouple from underlying system (be it Jini, J2EE, etc.)
• Spring can do remoting without API via exporters on server side, proxies on client side (talk to exporters)

Summary: I like the idea of doing a Grid overlay of federated databases. This is something that I'm going to need to explore more on my own. Posted by andyr at June 29, 2005 12:44 PM