Distributed Computing

I used the texts:

• Java Network Programming by Elliotte Rusty Harold,
• Client/Server Programming with Java and CORBA by Robert Orfali and Dan Harkey,
• Java in a Nutshell, Edition 1, Chapter 7 online and
• the Core Java Volume II, Chapter 3

• Nutshell, ed 1, Examples 7-*,
• Java FAQ

Networking - Introduction

Java provides cross-platform abstractions for simple networking operations, including connecting and retrieving files by using Web protocols and creating basic Unix-like sockets.

Used in conjunction with I/O streams, reading and writing files over the net becomes very easy.

Restrictions: Java applets cannot read or write from the disk on the machine that is running them. An applet can only establish a socket connection to the server on which it resides. Thus, an applet cannot be used as the server program.

Classes for networking are in package java.net

Very very simple starting out

• Start telnet
• For the host field, type time-a.timefreq.bldrdoc.gov
• For the port field, type 13 (terminal type doesn't matter)

I got back:
50914 98-04-11 22:10:07 50 0 0 50.0 UTC(NIST) *
This is a simple example of connecting to a service (the National Institute of Standards and Technology in Boulder, Colorado) and receiving information (a measurement of a Cesium atomic clock).

Before jumping into client/server programming for Java, one needs to understand some basics: Hosts, Ports, URLs and Sockets.

Ports

As a general (but far from absolute) rule each computer only has one Internet address. Addresses would be all one needs if each computer did no more than one thing at a time. However, on a given computer, there may be multiple ftp sessions, a few web connections, and a chat program all running at the same time.

To make this possible the computer's network interface is logically subdivided into 65,536 different ports. This is an abstraction. A port does not represent anything physical like a serial or parallel port. However as data traverses the Internet in packets, each packet carries not only the address of the host but also the port on that host to which it's aimed. The host is responsible for reading the port number from each packet it receives to decide which program should receive that chunk of data.

On Unix systems you must be root to listen for connections on ports between 1 and 1024. Anyone can listen for connections on ports of 1025 to 65,535 as long as the port is not already occupied. (No more than one program can listen on a given TCP port at the same time.) However on Windows NT, Windows 95, and the Mac any user can listen to any port. No special privileges are required.

Some examples (from "Java Network Programming", page 20). On UNIX machines a fairly complete listing of assigned ports is stored in the file etc/services

PROTOCOL	PORT	ENCODING	PURPOSE
echo	7	tcp/udp	Echo is a test protocol used to verify that two machines are able to connect by having one echo back the other's input
ftp-data	20	tcp	FTP uses two well known ports. This port is used to transport files
ftp	21	tcp	This port is used to send FTP commands like "put" and "get"
telnet	23	tcp	Telnet is a protocol used for interactive, remote command-line sessions
smtp	25	tcp	The "Simple Mail Transfer Protocol" is used to send email between machines
whois	43	tcp	A simple directory service for internet network administrators
finger	79	tcp	Gets information about a user or users
http	80	tcp	HyperText transfer Protocol the underlying protocol of the WWW

Any remote host can connect to a server that's listening on a port below 1024. Furthermore, multiple simultaneous connections may be made to a remote host on a remote port. For example, a high volume web server listening on port 80 may be processing several dozen connections at the same time, all connected to port 80.

In short, no more than one process on the local host can use a port at one time. However many remote hosts may connect to the same remote port.

Protocols and Internet Standards

If you need detailed information about any protocol, the definitive source is the collection of Internet drafts and requests for comments (RFCs). For a table of selected RFC, see "Java Network Programming", page 29, or Proposed Standards and Informational Material and another

Internet Addresses

Since humans have trouble remembering numbers like this, these addresses are mapped to names like "plasticman.ecst.csuchico.edu". However it's the numeric address that's fundamental, not the name.

Java's java.net.InetAddress class retrieves such addresses. Among others it contains methods to convert numeric addresses to host names and host names to numeric addresses. For example: getByName(String host) determines the IP address of a host, given the host's name, getHostAddress() returns the IP address string "%d.%d.%d.%d"

For those not familiar with UNIX, before jumping into the java program, you might want to check the address to see if it is up and running.
nslookup machinename
or
ping machinename
For example:

$ nslookup plasticman.ecst.csuchico.edu
Server:  velveeta.ecst.csuchico.edu
Address:  132.241.3.57
 
Name:    plasticman.ecst.csuchico.edu
Address:  132.241.2.90
 
$ ping 132.241.2.90
132.241.2.90 is alive

The InetAddress class is a little unusual in that it doesn't have any public constructors. (See factory method design pattern) Instead you pass the host name or string format of the dotted quad address to the static InetAddress.getByName() method like this:

try {
  InetAddress pip = InetAddress.getByName("pip.ecst.csuchico.edu");
  InetAddress mycin = InetAddress.getByName("132.241.7.225");
}
catch (UnknownHostException e) {
  System.err.println(e);
}  

URLs

http://www.javasoft.com/
file:///Macintosh%20HD/Java/Docs/JDK%201.1.1%20docs/api/java.net.InetAddress.html#_top_
http://www.macintouch.com:80/newsrecent.shtml
ftp://ftp.info.apple.com/pub/
mailto:elharo@sunsite.unc.edu
telnet://utopia.poly.edu

• the protocol
• the host
• the port
• the file
• the ref, section, or anchor

The java.net.URL class

There are constructors to create new URLs and methods to parse the different parts of a URL. However the heart of the class are the methods that allow you to get an InputStream from a URL so you can read data from a server.

The URL class is closely tied to protocol and content handlers. The objective is to separate the data being downloaded from the protocol used to download it. The protocol handler is responsible for communicating with the server, that is moving bytes from the server to the client. It handles any necessary negotiation with the server and any headers. Its job is to return only the actual bytes of the data or file requested. The content handler takes those bytes and translates them into some kind of Java object such as an InputStream or ImageProducer.

When you construct a URL object, Java looks for a protocol handler that understands the protocol part of the URL such as "http" or "mailto". If no such handler is found, the constructor throws a MalformedURLException. The exact protocols that are supported vary from implementation to implementation though http and file are supported pretty much everywhere. Sun's JDK 1.1 understands ten:

• file
• ftp
• gopher
• http
• mailto
• appletresource
• doc
• netdoc
• systemresource
• verbatim

Example 7.1 (in Nutshell Ed1 book) shows an simple example of creating a URL from a specified WWW address and then downloading its contents with the getcontent() method.

// This example is from the book _Java in a Nutshell_ by David Flanagan.
// Written by David Flanagan.  Copyright (c) 1996 O'Reilly & Associates.
// You may study, use, modify, and distribute this example for any purpose.
// This example is provided WITHOUT WARRANTY either expressed or implied.

import java.awt.*;
import java.io.*;
import java.net.*;

// The fetch() method in this class only works for fetching text/plain 
// data.  If you specify a file: URL, you may well need to specify a
// file that ends with a .txt extension so that the internal content
// handlers can tell it is a plain text file.  The standard Java 
// distribution doesn't contain content handlers for other types (such
// as text/html), and this application exits with an exception if it
// doesn't recognize the type or doesn't know how to load the type.
// The fetchimage() method works for .gif and a few other common image
// formats for which content handlers have been written.  See the
// FetchImageTest class for a demonstration of the fetchimage() method
// defined here.
//
// This class serves to demonstrate the URL.getContent() method.  In
// general, however, there are much better ways to load files and images
// over the net.  See Applet.getImage() for example:
public class Fetch {
    // Get the contents of a URL and return it as a string.
   public static String fetch(String address) 
       throws MalformedURLException, IOException 
    {
        URL url = new URL(address);
        return (String) url.getContent();
    }

    // Get the contents of a URL and return it as an image
    public static Image fetchimage(String address, Component c) 
        throws MalformedURLException, IOException 
    {
        URL url = new URL(address);
        return c.createImage((java.awt.image.ImageProducer)url.getContent());
    }

    // Test out the fetch() method.
    public static void main(String[] args) 
        throws MalformedURLException, IOException 
    {
        System.out.println(fetch(args[0]));
    }
}

Note their note: this example is intended only to demonstrate the use of URL.getContent(). In general, this is not the best way to load text or image files over the net. See for example: Applet.getImage(), or java.awt.Toolkit.createImage
public Image createImage(byte imagedata[])

My aside: be cautious of using actions for class names (Fetch). Their Fetch is a class which has a couple of methods for fetching (images and text). Perhaps the class should be called something like URLContentFetcher rather than Fetch

GetURLInfo.java (example 7-2) demonstrates how to obtain more information about a URL, and how to have more control over downloading the contents of the URL. URLInfoGetter rather than GetURLInfo as class name?

net API

// This example is from the book _Java in a Nutshell_ by David Flanagan.
// Written by David Flanagan.  Copyright (c) 1996 O'Reilly & Associates.
// You may study, use, modify, and distribute this example for any purpose.
// This example is provided WITHOUT WARRANTY either expressed or implied.

import java.net.*;
import java.io.*;
import java.util.*;

public class GetURLInfo {
    public static void printinfo(URLConnection u) throws IOException {
        // Display the URL address, and information about it.
        System.out.println(u.getURL().toExternalForm() + ":");
        System.out.println("  Content Type: " + u.getContentType());
        System.out.println("  Content Length: " + u.getContentLength());
        System.out.println("  Last Modified: " + new Date(u.getLastModified()));
        System.out.println("  Expiration: " + u.getExpiration());
        System.out.println("  Content Encoding: " + u.getContentEncoding());
        
        // Read and print out the first five lines of the URL.
        System.out.println("First five lines:");
        DataInputStream in = new DataInputStream(u.getInputStream());
        for(int i = 0; i < 5; i++) {
            String line = in.readLine();
            if (line == null) break;
            System.out.println("  " + line);
        }
    }
    
    // Create a URL from the specified address, open a connection to it,
    // and then display information about the URL.
    public static void main(String[] args) 
        throws MalformedURLException, IOException
    {
        URL url = new URL(args[0]);
        URLConnection connection = url.openConnection();
        printinfo(connection);
    }
}

More examples are found on the tutorial for Java Network Programming (mirrored below)

There are four constructors in the java.net.URL class. All can throw MalformedURLExceptions.

 public URL(String u) 
               throws MalformedURLException
 public URL(String protocol, String host, String file) 
               throws MalformedURLException
 public URL(String protocol, String host, int port, String file) 
               throws MalformedURLException
 public URL(URL context, String u) 
               throws MalformedURLException

  URL u = null;
  try {
    u = new URL("http://www.poly.edu/schedule/fall97/bgrad.html#cs");
  }
  catch (MalformedURLException e) {
  
  }

  URL u = null;
  try {
    u = new URL("http", "www.poly.edu", "/schedule/fall97/bgrad.html#cs");
  }
  catch (MalformedURLException e) {
  
  }

  URL u = null;
  try {
    u = new URL("http", "www.poly.edu", 80, "/schedule/fall97/bgrad.html#cs");
  }
  catch (MalformedURLException e) {
  
  }

However when this same page is loaded from
http://sunsite.lanet.lv/javafaq/course/week12/07.html,
then the link to "08.html" refers to
http://sunsite.lanet.lv/javafaq/course/week12/08.html
instead.

The fourth constructor above creates URLs relative to a given URL. For example,

  URL u1, u2;
  try {
    u1 = new URL("http://sunsite.unc.edu/javafaq/course/week12/07.html");
    u2 = new URL(u1, "08.html");
  }
  catch (MalformedURLException e) {
  
  }

Parsing URLs

 public String getProtocol()
 public String getHost()
 public int getPort() 
 public String getFile()
 public String getRef()

  try {
    URL u = new URL("http://www.poly.edu/schedule/fall97/bgrad.html#cs");
    System.out.println("The protocol is " + u.getProtocol());
    System.out.println("The host is " + u.getHost());
    System.out.println("The port is " + u.getPort());
    System.out.println("The file is " + u.getFile());
    System.out.println("The anchor is " + u.getRef());
  }
  catch (MalformedURLException e) {
  
  }

If the ref doesn't exist, it's just null, so watch out for NullPointerExceptions. Better yet, test to see that it's non-null before using it.

Finally if the file is left off completely, e.g. http://www.javasoft.com, then it's set to "/".

Reading Data from a URL

import java.net.*;
import java.io.*;

public class Webcat {

  public static void main(String[] args) {

    for (int i = 0; i < args.length; i++) {
      try {
        URL u = new URL(args[i]);
        InputStream is = u.openStream();
        InputStreamReader isr = new InputStreamReader(is);
        BufferedReader br = new BufferedReader(isr);
        String theLine;
        while ((theLine = br.readLine()) != null) {
          System.out.println(theLine);
        }
      }
      catch (MalformedURLException e) {
        System.err.println(e);
      } 
      catch (IOException e) {
        System.err.println(e);      
      } 
    }
  }
}

Sockets

Datagrams range in size from a few dozen bytes to about 60,000 bytes. Anything larger than this, and often things smaller than this, needs to be split into smaller pieces before it can be transmitted. The advantage is that if one packet is lost, it can be retransmitted without requiring redelivery of all other packets.

The socket represents a reliable connection for the transmission of data between two hosts. It isolates you from the details of packet encodings, lost and retransmitted packets, and packets that arrive out of order.

Socket objects are used to establish connections. They are endpoints of communication links between processes. They appear as file descriptors to a process, so data can be exchanged with another process by transmitting or receiving through the socket.

The type of socket used, describes the way the data is transferred through the socket.

• Stream Sockets (TCP) are a connection-oriented service where data can be transfered without record boundaries. If the connection between processes is broken, the processes will be informed.

• Datagram sockets (UDP "User Datagram Protocol" sometimes called "unreliable datagram protocol") a connectionless transport service across an IP network, is more efficient than TCP, but reliability is not guaranteed. Delivery is not guaranteed; packets may be duplicated, lost , or arrive in a different order to that in which they are sent.

  (Use when you want low-overhead communications of non-critical data and when a stream model of communication is not necessary)

  Both TCP and UDP split data into packets called datagrams. However TCP includes extra headers in the datagram to enable retransmission of lost packets and reassembly of packets into the correct order if they arrive out of order. UDP does not provide this. If a UDP packet is lost, it's lost. It will not be retransmitted. Similarly, packets appear in the receiving program in the order they were received, not necessarily in the order they were sent.

  Given these disadvantages you may well wonder why anyone would prefer UDP to TCP. The answer is speed. UDP can be up to three times faster than TCP; and there are many applications for which reliable transmission of data is not nearly as important as speed. For example lost or out of order packets may appear as static in an audio or video feed, but the overall picture or sound could still be intelligible.

  Telephone vs. snail mail analogy.

  Protocols that use UDP include NFS, FSP, and TFTP.

  I will not be covering UDP except to provide the following:

  Java's support for UDP is contained in two classes, java.net.DatagramSocket and java.net.DatagramPacket. A DatagramSocket is used to send and receive DatagramPackets. Since UDP is connectionless, streams are not used. You must fit your data into packets of no more than about 60,000 bytes. You can manually split the data across multiple packets if necessary.

  The DatagramPacket class is a wrapper for an array of bytes from which data will be sent or into which data will be received. It also contains the address and port to which the packet will be sent.

  The DatagramSocket class is a connection to a port that does the sending and receiving. Unlike TCP sockets, there is no distinction between a UDP socket and a UDP server socket. The same DatagramSocket can both and receive. Also unlike TCP sockets, a DatagramSocket can send to multiple, different addresses. The address to which data goes is stored in the packet, not in the socket.

Back to Sockets in general: all of this is transparent to the Java programmer. The host's native networking software transparently handles the splitting of data into packets on the sending end of a connection, and the reassembly of packets on the receiving end.

Instead,the Java programmer is presented with a higher level abstraction called a socket. Specifically, for networking applications beyond the URL and URLConnection, Java uses the classes Socket and ServerSocket.

There are four fundamental operations a socket performs. These are:

1. Connect to a remote machine
2. Send data
3. Receive data
4. Close the connection

Java and Sockets

There are four public constructors in the Socket class. (There are also two protected constructors and two deprecated constructors.) You need to at least specify the remote host and port you want to connect to. The host may be specified as either a string like "pip.ecst.csuchico.edu" or as an InetAddress object. The port should be an int between 1 and 65535. For that matter it should be a particular one. You need to know the port just as much as the hostname.

SocketTest.java with 5.0 Scanner use. Below is older version with InputStreamReader wrapped in a BufferedReader

/*
 * Cay S. Horstmann & Gary Cornell, Core Java
 * Published By Sun Microsystems Press/Prentice-Hall
 * Copyright (C) 1997 Sun Microsystems Inc.
 * All Rights Reserved.
 *
 * Permission to use, copy, modify, and distribute this 
 * software and its documentation for NON-COMMERCIAL purposes
 * and without fee is hereby granted provided that this 
 * copyright notice appears in all copies. 
 * 
 * THE AUTHORS AND PUBLISHER MAKE NO REPRESENTATIONS OR 
 * WARRANTIES ABOUT THE SUITABILITY OF THE SOFTWARE, EITHER 
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE 
 * IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 
 * PARTICULAR PURPOSE, OR NON-INFRINGEMENT. THE AUTHORS
 * AND PUBLISHER SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED 
 * BY LICENSEE AS A RESULT OF USING, MODIFYING OR DISTRIBUTING 
 * THIS SOFTWARE OR ITS DERIVATIVES.
 */
 
/**
 * @version 1.10 27 Jun 1997
 * @author Cay Horstmann
 */

import java.io.*;
import java.net.*;
 
class SocketTest
{  public static void main(String[] args)
   {  try
      {  // Socket t = new Socket("time-a.timefreq.bldrdoc.gov", 13);

         Socket t = new Socket("132.163.4.101", 13);

         BufferedReader is = new BufferedReader
            (new InputStreamReader(t.getInputStream()));
         boolean more = true;
         while (more)
         {  String str = is.readLine();
            if (str == null) more = false;
            else
               System.out.println(str);
         }
         
      } 
      catch(IOException e) 
      { System.out.println("Error" + e); }
   }
}

The Socket() constructors do not just create a Socket object. They also attempt to connect the underlying socket to the remote server. All the constructors throw an IOException if the connection can't be made for any reason.

Note that, just like with URLs, sending and receiving data is accomplished with output and input streams. There are methods to get an input stream for a socket and an output stream for the socket. Exactly what the data you send and receive means often depends on the protocol.

Most of the time you'll want to chain the InputStream to some other input stream or reader class to more easily handle the data (as was done in one line above and explicitly in the next example)

For example, the following code fragment connects to the daytime server on port 13 of sunsite.unc.edu, and displays the data it sends.

    try {
      Socket s = new Socket("sunsite.unc.edu", 13);
      InputStream is = s.getInputStream();
      InputStreamReader isr = new InputStreamReader(is);
      BufferedReader br = new BufferedReader(isr);
      String theTime = br.readLine();
      System.out.println(theTime);
    }
    catch (IOException e) {
      return (new Date()).toString();
    }

You cannot just connect to any port on any host. The remote host must actually be listening for connections on that port. You can use the constructors to determine which ports on a host are listening for connections.

Harold, in "Java Network Programming" and his tutorial provide the following:

import java.net.*;
import java.io.IOException;


public class PortScanner {

  public static void main(String[] args) {
    for (int i = 0; i < args.length; i++) {
      try {
        InetAddress ia = InetAddress.getByName(args[i]);
        scan(ia);
      }
      catch (UnknownHostException e) {
        System.err.println(args[i] + " is not a valid host name.");
      }
    }
  }

  public static void scan(InetAddress remote) {
    // Do I need to synchronize remote?
    // What happens if someone changes it while this method
    // is running?

    String hostname = remote.getHostName();
    for (int port = 0; port < 65536; port++) {
     try {
        Socket s = new Socket(remote, port); 
        System.out.println("A server is listening on port " + port
         + " of " + hostname);
        s.close();
      }
      catch (IOException e) {
        // The remote host is not listening on this port
      }
    }
  }

  public static void scan(String remote) throws UnknownHostException {

    // Why throw the UnknownHostException? Why not catch it like I did
    // in the main() method?
    InetAddress ia = InetAddress.getByName(remote);
    scan(ia);
  }
}

Warning: Pointing PortScanner at a machine you do not own would generally be considered a hostile act by the owner of the machine.

Next: Client Server examples. Use Socket and ServerSocket to make Client Server tools.