Distributed Computing Environment
A Project Report
Submitted in partial fulfilment of
the requirements for the award of the degree of
Master of Technology
in
Computer Science and Engineering
by
Dains Raj Antony
M105103
Department of Computer Science & Engineering
College of Engineering Trivandrum
Kerala - 695016
2010-11
Abstract
The aim of this project is to harvest the power of distributed computing system to perform
heavy operations. A distributed computer system consists of multiple software components
that are on multiple computers, but run as a single system. The computers that are in a
distributed system can be physically close together and connected by a local network, or they
can be geographically distant and connected by a wide area network. A distributed system
can consist of any number of possible configurations, such as mainframes, personal computers,
workstations, minicomputers, and so on. The objective is to create a programming library in
java so that programs written according to contracts specified by this library could easily be
distributed across client nodes in a network. Each node in the network will be running a client
and there will be a central server that dispatches tasks to the client, a protocol is to be defined
for communication between server and clients for transferring code and dependencies. Two way
interactive communications will be used to minimize the amount of data transfer. This work
is a small and modest part of a quite complex system.
[attachment=8652]
1 Introduction
Distributed computing is a field of computer science that studies distributed systems. A dis-
tributed system consists of multiple autonomous computers that communicate through a com-
puter network. The computers interact with each other in order to achieve a common goal. A
computer program that runs in a distributed system is called a distributed program.Distributed
computing also refers to the use of distributed systems to solve computational problems. In
distributed computing, a problem is divided into many tasks, each of which is solved by one
computer.
Distributed systems are networked computers operating with same processors,ie In dis-
tributed computing, each processor has its own private memory (distributed memory). In-
formation is exchanged by passing messages between the processors.Figure 1. is a schematic
view of a typical distributed system the system is represented as a graph in which each node
(vertex) is a computer and each edge (line between two nodes) is a communication link. Each
computer has its own local memory, and information can be exchanged only by passing messages
from one node to another by using the available communication links.
Figure 1: A sample Schematic diagram of a Distributed System
There are two main reasons for using distributed systems and distributed computing. First,
the very nature of the application may require the use of a communication network that connects
several computers. For example, data is produced in one physical location and it is needed in
another location.
Second, there are many cases in which the use of a single computer would be possible in
principle, but the use of a distributed system is beneficial for practical reasons. For example, it
may be more cost-efficient to obtain the desired level of performance by using a cluster of several
low-end computers, in comparison with a single high-end computer. A distributed system can
be more reliable than a non-distributed system, as there is no single point of failure. Moreover,
a distributed system may be easier to expand and manage than a monolithic uniprocessor
system.
5
In our project we are mainly concentrating on the second application ie,we are developing a
Distributed Computing Environment. which is independent of the topology of the network and
it can be scaled to the desired level with a minimum cost compared to that of a single high-end
system.A sample Environment is shown in Figure 2 .
Figure 2: This figure shows a sample topology of our Distributed computing Network.
Here we are using the processing power of the nodes in the network to solve a big task.For
this purpose there is a server which controls all the activities in the network.A user can submit
a job to the server and the server will identify the tasks and it then dispatches all the tasks to
different nodes in the network.
6
2 Technologies Used
The project is completely implemented in java. The main advantage of java is that it is
a platform independent language.since java is platform independent you don’t have to worry
about the platform of the distributed clients,ie in our Distributed computing Environment the
client node can be of any platform the only thing it needed is a JVM.The main technologies
used in this project are:
2.1 Reflection
Reflection can be used for observing and/or modifying program execution at runtime. A
reflection-oriented program component can monitor the execution of an enclosure of code and
can modify itself according to a desired goal related to that enclosure. This is typically accom-
plished by dynamically assigning program code at runtime.
Reflection can also be used to adapt a given program to different situations dynamically. For
example, consider an application that uses two different classes X and Y interchangeably to
perform similar operations. Without reflection-oriented programming, the application might be
hard-coded to call method names of class X and class Y. However, using the reflection-oriented
programming paradigm, the application could be designed and written to utilize reflection in
order to invoke methods in classes X and Y without hard-coding method names.
A language supporting reflection provides a number of features available at runtime that
would otherwise be very obscure or impossible to accomplish in a lower-level language. Some
of these features are the abilities to:
1. Discover and modify source code constructions (such as code blocks, classes, methods,
protocols, etc.) as a first-class object at runtime.
2. Convert a string matching the symbolic name of a class or function into a reference to or
invocation of that class or function.
3. Evaluate a string as if it were a source code statement at runtime.
2.2 Sockets
A socket is one endpoint of a two-way communication page link between two programs running
on the network. A socket is bound to a port number so that the TCP layer can identify
the application that data is destined to be sent.The socket is the software abstraction used to
represent the “terminals”of a connection between two machines. For a given connection, there’s
a socket on each machine, and you can imagine a hypothetical “cable”running between the two
machines with each end of the “cable”plugged into a socket. Of course, the physical hardware
and cabling between machines is completely unknown. The whole point of the abstraction is
that we don’t have to know more than is necessary.
The java.net package in the Java platform provides a class, Socket, that implements one
side of a two-way connection between your Java program and another program on the network.
The Socket class sits on top of a platform-dependent implementation, hiding the details of any
particular system from your Java program. By using the java.net.Socket class instead of
relying on native code, your Java programs can communicate over the network in a platform-
independent fashion.
7
Additionally, java.net includes the ServerSocket class, which implements a socket that
servers can use to listen for and accept connections to clients.
8
3 Components in the System
3.1 Task
In Computer Programming there are two types of parallelism one is called Implicit paral-
lelism and the other one is called Explicit parallelism implicit parallelism is a characteristic of
a programming language that allows a compiler or interpreter to automatically exploit the par-
allelism inherent to the computations expressed by some of the language’s constructs. A pure
implicitly parallel language does not need special directives, operators or functions to enable
parallel execution.
explicit parallelism is the representation of concurrent computations by means of primitives
in the form of special-purpose directives or function calls. Most parallel primitives are related
to process synchronization, communication or task partitioning
In our system, a program is explicitly parallelized by the programmer using predefined con-
structs. The most basic unit of work that can be parallelized is defined as a ̈ask ̈ In the
t .
Java environment that we use, this is represented by a class that inherits from an interface
called Task. This Task, when processed subject to predefined contracts, is executed using the
distributed power of the system. The Task interface contains a method:
void execute (Map args);
All the processing that has to be done by the task goes into this execute method. The
method accepts a single argument args which is a Map of key/value pairs. This way, we can
send a varying number of arguments to different Tasks that have different requirements and
different inputs.
3.2 TaskManager
TaskManager is one of the key components of the system that actually performs the distri-
bution of tasks among the different nodes. It run on the server. The principal method of this
class is:
void execute (String className, Map args);
This method accepts a class name and a Map of arguments to be input to the process.
The TaskManager then goes on to find a free node in the system. The TaskManager keeps a
database of which all nodes are free and busy at the moment. This database is kept up-to-date
with each Task invocation and completion. Priority is given to free nodes that already has the
application installed. If no such node is available, any free node is selected and the application
is first transferred to the node with the help of the Installer entity. The list of nodes and
the applications installed at each node is kept in the database owned by the Installer. Once
the client machine is ready, the Client process is invoked with the class name and the list of
arguments.
3.3 Client
Each processing node in the system runs a component process called the Client. It is this
process to which the TaskManager delegates the responsibility of running a Task. The principal
method of the class is:
9
void execute (String className, Map args);
This method is invoked by the TaskManager on the Client of the free node that it finds.
The Client then uses Java reflection to dynamically load the requested Task and invoke it’s
execute method. The Map of input parameters is also passed into the method as an argument.
The Client also has a role in the installation of the application on this machine. Whenever
a Task is to be first run on a node, the application containing the Task has to be installed on
this. This activity is controlled by the Installer module and the Client assists the process
of setting the application up on the node it represents. The signature of the concerned method
is:
void install(String appId, Byte[] archiveData);
3.4 Installer
The Installer module runs on the server and it’s job is to facilitate the installation of new
applications on the server as well as in the nodes. It has the principal method:
void install (String archiveFilename);
It takes as input the path to a ZIP file containing the application jar as well as the support
libraries. It then extracts the file into the projects folder that holds all applications. The
ZipInputStream library in Java is used to extract the archive file. The Installer also assigns
an ID for the project and the project database is updated with this information.
When a Task is to be run on a node for the very first time, the application has to be installed
on the client node as well. This is also supervised by the Installer module. It commands the
Client module on the node to install the application after supplying it with the required files.
10
4 Communication
The system uses TCP/IP communication using sockets for the transfer of control and data
between the client as well as the server modules. The general structure of the application layer
datagram used is as shown in the Fig. 2.
Figure 1: Application layer datagram format.
The Data portion of the datagram is a variable length field whose length is defined in the Data
Length field. The structure of the Data field also differs significantly based on the Command
being sent. The basic commands used in the system are defined below.
4.1 INSTALL
Sent by the Installer module to the Client to initiate the installation of the application on
a remote node. The binary contents of the archive file to be installed is included in the Data
part of the datagram which is then used by the Client to do the setup.
4.2 EXECUTE
Sent by the TaskManager to initiate the execution of a Task on the client machine. The data
part contains the name of the Task to be invoked as well as the list of parameters in the form
of a Map.
4.3 DONE
This message is sent back from a Client node back to the TaskManager indicating the comple-
tion of a Task. This information is used by the TaskManager to keep the free/busy information
up-to-date.
11
References
[1] Y Feng A Ripke, A R Allen and S C Allison. “Distributed computing using channel com-
munications and java”. Communicating Process Architectures 2000, pages 49–63, January
2000.
[2] Jim Farley. “Java distributed computing”. page 384, January 1998.
[3] http : //en.wikipediawiki/Distributedc omputing.
12
