Lightweight Bartering Grid

lbg.protocol.data
Class GridDataManager

java.lang.Object
  extended by lbg.protocol.data.GridDataManager

public class GridDataManager
extends java.lang.Object

Handles GridData. The data manager (hereafter GDM) can host and retrieve GridData files, and cache them for later use.
It is a merge of the previous separate Peer- and ResourceDataManager, grouped for convenience and ease of use.
A lot of work was done to make this a normal object instead of a singleton (a choice that was initially lead by the use of Azureus as BitTorrent engine). It is now much more convenient to use but could be subject to some documented side effects.
Related logger: lbg.data
Warnings:

Author:
Xavier Dalem
See Also:
Concept of Working Set, Warning about use of user.home in conjunction with this class

Field Summary
static int HOST_FTP
          Download a GridData and host it on an FTP Server.
static int HOST_TORRENT_SEED
          Download a TorrentGridData and act as a new seed.
static int HOST_TORRENT_TRACK
          Download a GridData and seed & track it as a TorrentGridData.
static int INFINITE_CAPACITY
          Denotes that the GDM cache has an infinite capacity.
static int NUM_SHARE_OPTIONS
          Number (counting 1) of elements in a valid GridData download options array.
static int SHARED_FTP
          Index of the FTPGridData in the GridData download options array.
static int SHARED_INLINE
          Index of the InlineGridData in the GridData download options array.
static int SHARED_TORRENT
          Index of the TorrentGridData in the GridData download options array.
 
Constructor Summary
GridDataManager(java.lang.String hostname, java.lang.String cacheDir, int btDataPort, int ftpDataPort, java.lang.String trackerHost, int trackerPort, int capacity, int sharePreferences, java.lang.String statsDirectory)
          Creates a GridDataManager.
 
Method Summary
 boolean checkWorkingSet(GridData[] workingSet)
          Checks if the passed GridData's would fit in the Working Set.
 GridData extract(GridData[] data)
          Selects the preferred download option out of an array of possibilities.
 GridData[] extract(GridData[][] datas)
          Extracts an array of data downloading options.
 java.io.File getFile(GridData data)
          Store data in cache and get the associated File.
Data saved this way is never considered as local.
 java.io.File getFile(GridData[] data)
          Convenience method to automate extraction of data when downloading
protected  Metadata<java.lang.String> getMetadata(java.lang.String id)
           
 int getSeedCount(java.lang.String dataID, boolean countPeers)
          Get the number of seeds participating in a BitTorrent download for a particular file
 boolean hasDefaultPolicy()
          Detects whether the manager was configured with a default policy or not.
static byte[] hash(java.io.File f)
          Hashes a file using the default algorithm ("SHA-256").
static byte[] hash(java.io.File f, java.lang.String algorithm)
          Hashes a file using the supplied algorithm.
static java.lang.String hashToString(byte[] hash)
          Returns a human-readable representation of a hash.
 GridData[] host(GridData data, boolean isLocal)
          Hosts given GridData with default sharing policy.
 GridData[] host(GridData data, boolean isLocal, int prefs)
          Hosts given GridData.
protected  boolean isCached(java.lang.String id)
           
 void restartFTPServer()
          Restart the FTP server, forcing reload of the configuration.
 void seed(GridData[][] datasOptions)
          Ensures that all given data's are being seeded.
 GridDataManager setFTPConfig(int port)
          Sets the FTP server configuration.
If a server is already running, it will need to be restarted for the changes to take effect.
 GridDataManager setFTPConfig(int port, java.lang.String user, java.lang.String password)
          Sets the FTP server configuration.
If a server is already running, it will need to be restarted for the changes to take effect.
 boolean setWorkingSet(GridData[] workingSet)
          Sets the working set.
 boolean setWorkingSet(GridData[][] workingSet)
          Sets the working set.
 java.lang.String toString()
           
 void waitFor(GridData[] requirements)
          Blockingly wait for all required data's to be present in cache.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

INFINITE_CAPACITY

public static final int INFINITE_CAPACITY
Denotes that the GDM cache has an infinite capacity.

See Also:
Constant Field Values

HOST_TORRENT_SEED

public static final int HOST_TORRENT_SEED
Download a TorrentGridData and act as a new seed. This behavior improves the swarm of a torrent but the node using this mode doesn't change anything to the GridData itself. Should be used as a mask for download preferences

See Also:
host(GridData, boolean, int), Constant Field Values

HOST_TORRENT_TRACK

public static final int HOST_TORRENT_TRACK
Download a GridData and seed & track it as a TorrentGridData. Implies HOST_TORRENT_SEED. Should be used as a mask for download preferences

See Also:
HOST_TORRENT_SEED, host(GridData, boolean, int), Constant Field Values

HOST_FTP

public static final int HOST_FTP
Download a GridData and host it on an FTP Server. Should be used as a mask for download preferences

See Also:
host(GridData, boolean, int), Constant Field Values

SHARED_TORRENT

public static final int SHARED_TORRENT
Index of the TorrentGridData in the GridData download options array.

See Also:
Constant Field Values

SHARED_FTP

public static final int SHARED_FTP
Index of the FTPGridData in the GridData download options array.

See Also:
Constant Field Values

SHARED_INLINE

public static final int SHARED_INLINE
Index of the InlineGridData in the GridData download options array.

See Also:
Constant Field Values

NUM_SHARE_OPTIONS

public static final int NUM_SHARE_OPTIONS
Number (counting 1) of elements in a valid GridData download options array.

See Also:
Constant Field Values
Constructor Detail

GridDataManager

public GridDataManager(java.lang.String hostname,
                       java.lang.String cacheDir,
                       int btDataPort,
                       int ftpDataPort,
                       java.lang.String trackerHost,
                       int trackerPort,
                       int capacity,
                       int sharePreferences,
                       java.lang.String statsDirectory)
                throws GridException
Creates a GridDataManager.

Parameters:
hostname - Hostname of the server or client owning this manager.
cacheDir - Absolute path to cache directory, without trailing path separator (see java.io.File.separatorChar). The directory must be created and writable.
btDataPort - The port for TCP and UDP BitTorrent transfers. It won't be taken into account if an Azureus core was already started
ftpDataPort - Port for FTP data connections. Needed for FTPGridData downloads.
trackerHost - The BitTorrent tracker host (i.e. the local host). Set to null for no tracking capabilities.
trackerPort - The desired BitTorrent tracker port. Will be ignored if trackerHHost is null.
capacity - The number of supplying files this manager can handle. INFINITE_CAPACITY for no upper bound. See class description for details.
sharePreferences - Flags for setting how data given to host() will be shared. 0 if you don't plan to host anything.
statsDirectory - Where the cache hit stats should be saved. Set to null If you don't want to keep track of the stats.
Throws:
GridException - If the parameters are set incorrectly.
See Also:
PeerDataConfiguration.buildRcvPolicy(String)
Method Detail

hash

public static byte[] hash(java.io.File f)
                   throws GridException
Hashes a file using the default algorithm ("SHA-256").

Parameters:
f - The file to hash
Returns:
the hash in a byte array
Throws:
GridException - If hashing cannot be completed
See Also:
GridData.HASHING_ALGORITHM

hash

public static byte[] hash(java.io.File f,
                          java.lang.String algorithm)
                   throws GridException
Hashes a file using the supplied algorithm.

Parameters:
f - The file to hash
algorithm - The desired hashing algorithm, as per MessageDigest
Returns:
the hash in a byte array
Throws:
GridException - If hashing cannot be completed
See Also:
hash(File)

hashToString

public static java.lang.String hashToString(byte[] hash)
Returns a human-readable representation of a hash.

Parameters:
hash - The hash to print
Returns:
A String containing an hexadecimal representation of the hash (1 character per 4 bits)

toString

public java.lang.String toString()
Overrides:
toString in class java.lang.Object
Returns:
A string representation of this manager.

hasDefaultPolicy

public boolean hasDefaultPolicy()
Detects whether the manager was configured with a default policy or not.

Returns:
true if the manager has a default policy

isCached

protected boolean isCached(java.lang.String id)

getMetadata

protected Metadata<java.lang.String> getMetadata(java.lang.String id)

checkWorkingSet

public boolean checkWorkingSet(GridData[] workingSet)
Checks if the passed GridData's would fit in the Working Set. This method does not change the state of the GridDataManager in any way.

Parameters:
workingSet - The candidate working set
Returns:
true if the working set could fit.

setWorkingSet

public boolean setWorkingSet(GridData[][] workingSet)
Sets the working set. This call is not blocking.

Parameters:
workingSet - The new working set (an array of download options)
Returns:
false if working set is larger than cache or data couldn't be scheduled for download

setWorkingSet

public boolean setWorkingSet(GridData[] workingSet)
Sets the working set.

A Working Set is a set of GridData's that should stay in cache. When data has to be popped from cache, only those not in the WS will be candidate.

This call is not blocking, data will be transferred in the background.

Parameters:
workingSet - The new working set (an array of different data's). Set to null to remove working set management.
Returns:
false if working set is larger than cache or data couldn't be scheduled for download

waitFor

public void waitFor(GridData[] requirements)
             throws GridException
Blockingly wait for all required data's to be present in cache. Though blocking, the method will not hold the lock for the whole time.

The required data has to be in the current working set. Else, we can't be sure it will ever be downloaded. If the data was already known, the submitted one's local file copy is set.

This method doesn't actually download the data.

Parameters:
requirements - The data's to wait for
Throws:
GridException - if the active working set isn't included in the previously given working set.
See Also:
setWorkingSet(GridData[])

extract

public GridData[] extract(GridData[][] datas)
Extracts an array of data downloading options. The extraction is based on the preferences given at instantiation and available download types.
Warning: If preferences only list unavailable types, some or all entries in the returned array might be null. The only way to be sure to to always have a valid returned value is to ensure that the data preferences list all types.

Parameters:
datas - An array of arrays of options for downloading GridData's

extract

public GridData extract(GridData[] data)
Selects the preferred download option out of an array of possibilities. The selected option is

Returns:
The preferred option, or null if none matched.

host

public GridData[] host(GridData data,
                       boolean isLocal,
                       int prefs)
                throws GridException
Hosts given GridData.

Hosting some GridData is saving it locally and rendering it available for other peers / resources using different protocols depending on the flags.

Parameters:
data - The GridData to host.
isLocal - Whether the data is local to the GDM owner (true) or from an extern consumer (supplying: false). The cache policy for local data is to leave data in cache as long as it's not explicitly deleted. Cache for supplying data is generally limited.
prefs - Flags for setting how data given to host() should be shared (overrides the default policy if any).
Returns:
An array of GridData's, each type being accessed by the SHARE_* static indices. Depending on the flags, some or all of the elements might be null.
Throws:
GridException - if hosting failed (could be at numerous places depending on the flags)
See Also:
HOST_TORRENT_SEED, HOST_TORRENT_TRACK, HOST_FTP, SHARED_TORRENT, SHARED_FTP, extract(GridData[])

host

public GridData[] host(GridData data,
                       boolean isLocal)
                throws GridException
Hosts given GridData with default sharing policy.

Throws:
GridException - if the manager doesn't have a default policy
See Also:
host(GridData, boolean, int), extract(GridData[])

seed

public void seed(GridData[][] datasOptions)
          throws GridException
Ensures that all given data's are being seeded. If it's not the case, seed them. Given data's are assumed to be already downloaded.

Parameters:
datasOptions - An array of arrays of data transfer options for data's. In other words, there are datasOptions.length data's to check.
Throws:
GridException

getSeedCount

public int getSeedCount(java.lang.String dataID,
                        boolean countPeers)
Get the number of seeds participating in a BitTorrent download for a particular file

Parameters:
dataID - The identifier of the data to check for
countPeers - true if the BitTorrent Peers should also be counted. These could have no interesting pieces right now, but could become valuable sharers anyway.
Returns:
The number of seeds (and peers if asked)

getFile

public java.io.File getFile(GridData data)
                     throws GridException
Store data in cache and get the associated File.
Data saved this way is never considered as local. Peers wanting to share data should instead use host(GridData, boolean).
This call is blocking.

Parameters:
data - The data to store
Returns:
The file where the data was stored
Throws:
GridException - if the data wasn't successfully downloaded and stored.

getFile

public java.io.File getFile(GridData[] data)
                     throws GridException
Convenience method to automate extraction of data when downloading

Parameters:
data - An array of dowload options for a GridData (not an array of independent GridData's)
Throws:
GridException

restartFTPServer

public void restartFTPServer()
                      throws GridException
Restart the FTP server, forcing reload of the configuration.

Throws:
GridException

setFTPConfig

public GridDataManager setFTPConfig(int port)
                             throws GridException
Sets the FTP server configuration.
If a server is already running, it will need to be restarted for the changes to take effect. If no user information is given, it is either recovered from what was previously set or generated randomly.

Parameters:
port - FTP Requests port
Returns:
the current GDM, for convenience
Throws:
GridException - if the parameters are invalid
See Also:
restartFTPServer()

setFTPConfig

public GridDataManager setFTPConfig(int port,
                                    java.lang.String user,
                                    java.lang.String password)
                             throws GridException
Sets the FTP server configuration.
If a server is already running, it will need to be restarted for the changes to take effect.

Parameters:
port - FTP Requests port
user - User name
password - Associated password
Returns:
the current GDM, for convenience
Throws:
GridException - if the parameters are invalid
See Also:
restartFTPServer()

Lightweight Bartering Grid

Copyright (c) 2005-2008, Cyril Briquet, parts Xavier Dalem.