Difference between revisions of "Topology"

From MosixWiki
Jump to: navigation, search
(New page: TOPOLOGY(M7) MOSIX Description TOPOLOGY(M7) '''NAME''' TOPOLOGY - incorporating networking costs in MOSIX '''TUNING''' MOSIX can make ...)
 
 
(2 intermediate revisions by one user not shown)
Line 2: Line 2:
 
    
 
    
 
  '''NAME'''
 
  '''NAME'''
−
     TOPOLOGY - incorporating networking costs in MOSIX
+
     TOPOLOGY - incorporating networking overheads in MOSIX
 
    
 
    
 
  '''TUNING'''
 
  '''TUNING'''
−
     MOSIX can make better migration decisions when it knows the costs
+
     MOSIX can make better migration decisions when it has a good estimate of
−
     involved in running processes away from home.  The program tune, with its
+
     the overheads involved in running processes away from home: the program
−
    front-end tune_mosix, can be used to measure the 26 networking cost con-
+
    tune, with its front-end tune_mosix, can be used to measure 7 different
−
     stants between pairs of nodes.
+
     constants that reflect those overheads between any given two nodes.
−
 
+
     Since using tune can be tedious, a library of tuning-results for commonly
−
     Since using tune might be tedious, a library of tuning-results for com-
+
     used combinations of processors and networks is provided.  This library
−
     monly used combinations of processors and networks is provided.  This
+
     is in /etc/mosix/tunes, where each file contains 2 lines: the first
−
     library is in /etc/mosix/tunes, where each file contains 2 lines: the
+
     describes the computers and the network used to produce the results and
−
     first describes the computers and the network used to produce the results
+
     the second contains the 7 (space-separated) constants produced by tune.
−
     and the second contains the 26 (space-separated) constants produced by
+
   
−
    tune.
+
     Those constants depend on various factors such as network speed and
−
 
+
     latency, processor type, memory type, network card, whether a VPN layer
−
     The parameters that are taken into account include the network speed and
+
     is used on top of the IP protocol, etc.
−
     latency, processor type, memory type, network card, and most importantly,
+
−
    whether using a VPN layer on top of IP.
+
−
 
+
−
'''INTRODUCING TOPOLOGY'''
+
−
     Since the communication costs might not be uniform across the clus-
+
−
    ter/grid, the topology is defined as a set of tuning constants from a
+
−
    subset of nodes to the local node.  MOSIX supports up to 10 topologies,
+
−
    i.e., each node can define up to 10 different networking costs for dif-
+
−
    ferent sets of nodes in the cluster and/or grid.
+
 
      
 
      
−
     MOSIX comes with a built-in default topology of 1, that is a set of con-
+
'''INTRODUCING TOPOLOGY'''
−
     stants that applies uniformly to all nodes.  To override this default,
+
     The overheads of running a process away from its home may not be uniform
−
    the system-administrator needs to create the file: /etc/mosix/mostune.
+
    across the cluster or multi-cluster grid: the topology is therefore
−
    Once that file is created, MOSIX will automatically update its topology
+
    defined as a set of overhead constants measured between the local node
−
    within one minute.
+
     and a subset of other nodes.  MOSIX supports up to 10 topologies, allow-
−
 
+
     ing each node to define up to 10 sets of overhead constants, measured
−
    Each line in /etc/mosix/mostune should contain 29 (space-separated) con-
+
     between itself and different sets of nodes in the cluster and/or multi-
−
    stants: 26 for tuning and 3 for topological condition (see below), stat-
+
     cluster grid.
−
     ing to which node(s) should the corresponding set of 26 tuning constants
+
−
     refer (note that the last line may contain only 26 constants, in which
+
−
     case the last 3 are assumed 0).
+
 
    
 
    
−
     To select the costs for a given node, MOSIX scans the above conditions,
+
     MOSIX comes with a built-in single default topology - a set of pre-mea-
−
    starting with the first line and proceeding down the list until a condi-
+
     sured constants that applies uniformly to all nodes.  To override this
−
     tion is found that is satisfied by the given node (if no condition is
+
     default, the system-administrator needs to create the file:
−
     satisfied, the first line is selected).
+
    /etc/mosix/newtune (once that file is created or modified, MOSIX will
 +
    automatically update its topology within one minute).
 
    
 
    
 +
    Each line in /etc/mosix/newtune should contain 10 space-separated inte-
 +
    gers: 7 are the overhead constants (generated by tune_mosix) and 3 are
 +
    topological conditions (see below) that describe to which node(s) those
 +
    overhead constants apply (the last line can have only 7 constants, mak-
 +
    ing it unconditional).
 +
     
 +
    To decide which overhead constants apply for a given node, MOSIX scans
 +
    the above conditions, starting with the first line and proceeding down
 +
    the list until a condition is found that is satisfied by the given node
 +
    (if no condition is satisfied, the first line is selected).
 +
   
 
  '''TOPOLOGICAL CONDITIONS'''
 
  '''TOPOLOGICAL CONDITIONS'''
−
    Topological conditions are constructed from three constants (unsigned
+
      Topological conditions consist of three numbers (unsigned 32-bit inte-
−
    32-bit integers), named: FIRST, LAST, and FEATURES.  FIRST and LAST are
+
    gers), named: FIRST, LAST, and FEATURES.  FIRST and LAST are IP
−
     IP addresses (host order), while FEATURES is a bitmap.
+
     addresses, while FEATURES is a bitmap (1st symbol in /etc/mosix/features
 +
    is 1, 2nd symbol is 2, 3rd symbol is 4, etc.).  The IP addresses are rep-
 +
    resented as an unsigned integer in host order, so for example, IP address
 +
    123.45.67.89 is represented as:
 +
    ((123*256+45)*256+67)*256+89 = 2066563929
 
    
 
    
−
     In MOSIX, each node is associated with an IP address (IP) and a list of
+
     To test whether a given node satisfies a condition, we consider both its
−
    features associated with it ("F"). A condition is said to be satisfied
+
    IP address ("IP") and its features ("F"): the features are configured in
−
    by a node when:
+
    /etc/mosix/myfeatures (see below) and are 0 if that file does not exist.
 +
 
 +
    The table below covers all the 5 possible combinations of FIRST and LAST,
 +
    describing when a condition is satisfied by a given node:
 
    
 
    
 
     FIRST == LAST == 0      Always: unconditional
 
     FIRST == LAST == 0      Always: unconditional
Line 64: Line 71:
 
     FIRST > LAST > 0        (IP < FIRST || IP > LAST) && ~(F & FEATURES)
 
     FIRST > LAST > 0        (IP < FIRST || IP > LAST) && ~(F & FEATURES)
 
    
 
    
−
     The features of a node are listed in /etc/mosix/myfeatures by a (comma-
+
'''CONFIGURING FEATURES'''
−
    separated) list of symbols.  The only purpose of these symbols is to aid
+
     The features of a node are listed in the file /etc/mosix/myfeatures by a
−
     in constructing useful combinations of topological conditions. It is up
+
    comma-separated list of symbols, selected out of the 32 symbols in the
−
    to the grid/cluster administrators to assign values to the symbols that
+
    file /etc/mosix/features.
−
     are listed in /etc/mosix/features. While MOSIX comes with 32 predefined
+
 
−
     symbols, administrators could modify those symbols if necessary, provided
+
    These symbols have no particular meaning other than to aid in construct-
−
     they do the same throughout the cluster/grid.
+
     ing useful combinations of topological conditions. It is up to the multi-
 +
    cluster system-administrators to agree between them on conventional mean-
 +
     ings to those symbols.
 +
 
 +
     System-administrators are also allowed to modify those symbols if they
 +
     wish, provided that they keep /etc/mosix/features the same throughout the
 +
    multi-cluster Grid (if they do so, they must remember to restore that
 +
    file after upgrading to a new version of MOSIX).  
 
    
 
    
 
  '''SEE ALSO'''
 
  '''SEE ALSO'''
 
     mosix(7).
 
     mosix(7).
 
    
 
    
−
  MOSIX                              May 2006                             MOSIX
+
  MOSIX                              June 2008                             MOSIX

Latest revision as of 14:41, 22 February 2009

TOPOLOGY(M7)                   MOSIX Description                  TOPOLOGY(M7)
 
NAME
    TOPOLOGY - incorporating networking overheads in MOSIX
 
TUNING
    MOSIX can make better migration decisions when it has a good estimate of
    the overheads involved in running processes away from home: the program
    tune, with its front-end tune_mosix, can be used to measure 7 different
    constants that reflect those overheads between any given two nodes.
    Since using tune can be tedious, a library of tuning-results for commonly
    used combinations of processors and networks is provided.  This library
    is in /etc/mosix/tunes, where each file contains 2 lines: the first
    describes the computers and the network used to produce the results and
    the second contains the 7 (space-separated) constants produced by tune.
    
    Those constants depend on various factors such as network speed and
    latency, processor type, memory type, network card, whether a VPN layer
    is used on top of the IP protocol, etc.
   
INTRODUCING TOPOLOGY
    The overheads of running a process away from its home may not be uniform
    across the cluster or multi-cluster grid: the topology is therefore
    defined as a set of overhead constants measured between the local node
    and a subset of other nodes.  MOSIX supports up to 10 topologies, allow-
    ing each node to define up to 10 sets of overhead constants, measured
    between itself and different sets of nodes in the cluster and/or multi-
    cluster grid.
 
    MOSIX comes with a built-in single default topology - a set of pre-mea-
    sured constants that applies uniformly to all nodes.  To override this
    default, the system-administrator needs to create the file:
    /etc/mosix/newtune (once that file is created or modified, MOSIX will
    automatically update its topology within one minute).
 
    Each line in /etc/mosix/newtune should contain 10 space-separated inte-
    gers: 7 are the overhead constants (generated by tune_mosix) and 3 are
    topological conditions (see below) that describe to which node(s) those
    overhead constants apply (the last line can have only 7 constants, mak-
    ing it unconditional).
     
    To decide which overhead constants apply for a given node, MOSIX scans
    the above conditions, starting with the first line and proceeding down
    the list until a condition is found that is satisfied by the given node
    (if no condition is satisfied, the first line is selected).
   
TOPOLOGICAL CONDITIONS
     Topological conditions consist of three numbers (unsigned 32-bit inte-
    gers), named: FIRST, LAST, and FEATURES.  FIRST and LAST are IP
    addresses, while FEATURES is a bitmap (1st symbol in /etc/mosix/features
    is 1, 2nd symbol is 2, 3rd symbol is 4, etc.).  The IP addresses are rep-
    resented as an unsigned integer in host order, so for example, IP address
    123.45.67.89 is represented as:
    ((123*256+45)*256+67)*256+89 = 2066563929
 
    To test whether a given node satisfies a condition, we consider both its
    IP address ("IP") and its features ("F"): the features are configured in
    /etc/mosix/myfeatures (see below) and are 0 if that file does not exist.
 
    The table below covers all the 5 possible combinations of FIRST and LAST,
    describing when a condition is satisfied by a given node:
 
    FIRST == LAST == 0      Always: unconditional
 
    FIRST == 0; LAST != 0   (F & FEATURES) != 0
 
    LAST == 0; FIRST != 0   IP != FIRST && ~(F & FEATURES)
 
    0 < FIRST <= LAST       FIRST <= IP <= LAST || (F & FEATURES)
 
    FIRST > LAST > 0        (IP < FIRST || IP > LAST) && ~(F & FEATURES)
 
CONFIGURING FEATURES
    The features of a node are listed in the file /etc/mosix/myfeatures by a
    comma-separated list of symbols, selected out of the 32 symbols in the
    file /etc/mosix/features.
 
    These symbols have no particular meaning other than to aid in construct-
    ing useful combinations of topological conditions. It is up to the multi-
    cluster system-administrators to agree between them on conventional mean-
    ings to those symbols.
 
    System-administrators are also allowed to modify those symbols if they
    wish, provided that they keep /etc/mosix/features the same throughout the
    multi-cluster Grid (if they do so, they must remember to restore that
    file after upgrading to a new version of MOSIX). 
 
SEE ALSO
    mosix(7).
 
MOSIX                              June 2008                              MOSIX
Retrieved from "https://www.cs.huji.ac.il:80/wikis/MediaWiki/mosix/index.php?title=Topology&oldid=160"