Tag: cluster

Configuring DHCP and NAT in ODROID XU4 Cluster

michael

UPDATE – I have rebuilt this cluster to use Ubuntu 16.04. You can find updated instruction for Ubuntu 16.04 here. As was discussed in the network design post, we will set up the master node as a router to manage network traffic in and out of the cluster.  Before starting, ensure that all of the slave nodes have been powered down, that your home network is still connected directly to the open port on the cluster’s ethernet switch, that you have collected each node’s MAC address, and that the master node is powered up and you are logged into it via SSH. The first step is to explicitly set up the networking interfaces for both the eth0 and eth1 device on the master node. Note that by default the Ubuntu system that was installed on the master node treats eth0 a as requesting a DHCP lease on the network it is attached to. This is why it got an IP Read More …

Network Design for the Low Cost Cluster

michael

Our first task in building any cluster is to first design how it will be set up, most notably how the nodes will interact with each other. The cluster we will be building will have 4 nodes, one master node and three slaves. Each node will be connected to each other by the ethernet switch. However, we want the node-to-node communication to be it’s own network. This maximizes the throughput in the node-to-node communication, which is important for distributed computation, and also makes the cluster behave more like a single device to the external network. The benefit of doing this is that we can add or remove nodes to the cluster without a client ever knowing. However, this approach does present some challenges with how an external client (e.g., your laptop) will interact with the data analysis software, such as Hadoop, but we will deal with that later. My goal is really to create a “data analysis appliance”, so requiring any client Read More …

Building the ODROID XU4 Low Cost Cluster

michael

My shipment of ODROID XU4s that I ordered for my low cost cluster came into today. So I set out to assemble the cluster. My daughter “helped” me with this, making it a fun family activity. First thing I did was layout the nodes. Though I knew these board would be small, it did impress me just how small they were. The first task is to attach the eMMC drive to each node. Note that I eventually had to undo this as I later discovered I needed to flash the drives with the latest Ubuntu build. But given the amount of space the PCB standoffs five, removing and reattaching the eMMC drive is not hard. Then I attached the XU4s together with the PCB standoffs I bought. The short standoffs are for the “feet” to the stack, and the long ones to separate each node. Once I stacked all of the nodes, I attached the ethernet switch and the cabling. I Read More …