Setting up ESbox Using a Virtual Machine

This page covers the concepts and configuration for running ESbox on Windows and Mac OS X (or Linux x86/64) and hosting the Maemo SDK in a virtual machine.

(This does not apply to environments where Eclipse and the Maemo SDK are both inside a virtual machine. There is no special setup in those cases.)

• Overview
• Virtual Machine Engines Supported
• Virtual Machine Images
• Networking setup
• ESbox Build Machine Configuration

• ESbox Virtual Machine Behavior
• Clock synchronization
• PC-Connectivity Interaction
• Troubleshooting

Overview

The Maemo SDK, Scratchbox, and rootstraps only run on Linux/x86. If you want to use ESbox in Windows or MacOS X -- or even Linux/x86-64 -- then you can do this by hooking up ESbox with an appropriately configured virtual machine.

Recommended virtual machine images are provided by the maemovmware project. That project covers the configuration and installation issues specific to the virtual machine itself. (Note, although the project includes the term "vmware", the images (*.vmdk files) available there will work with all the supported engines.) This document covers configuring ESbox to use such images.

ESbox has the concept of Build Machines to distinguish the Eclipse host from the machine where C/C++ programs are built and Python programs are interpreted. For Linux/x86 hosts, the host can serve as a build machine. For other hosts, only a virtual machine can be a build machine.

ESbox will use an SSH connection with the virtual machine to perform builds, launch/debug applications, and configure Scratchbox installations and rootstraps, and a Samba connection to share project files between the machines.

An essential aspect of using ESbox with a VM is setting up shared folders. In order to use all the features in Eclipse, projects and the SDK rootstrap must be visible both to the host and the VM. See this document for an overview and setup instructions.

Virtual Machine Engines Supported

ESbox supports VMware, QEMU, and VirtualBox.

• VMware Workstation/Server 6.x (or Player 2.x) is supported and recommended for speed.

  NOTE: we have detected some problems using VMware Player 3.0. It seems that SSH dropouts are common in this version. VMware Player 2.x seems to work fine.

• VirtualBox 2.1 or newer is supported and recommended for freedom and consistency of behavior across operating systems.
• QEMU 0.9.0 and newer may be used for freedom and ease of setup, but is quite slow, even when accelerated.

Virtual Machine Images

ESbox runs best with a preconfigured "server" virtual machine image, which has a minimal footprint and is configured to provide the best interaction with ESbox.

These images may be used with any of the engines above.

The wizard File > New > Other > Maemo Installers > Maemo SDK Virtual Image will let you fetch and install the recommended images maintained by the Maemo SDK Virtual Machine project in concert with the Maemo Eclipse Integration project. These images come "bare", without any pre-installed Maemo SDK. ESbox supports installing these from the installer wizard.

Alternately, you can try a "desktop" image if you want a full environment pre-installed. (This contains ESbox inside. If you use that embedded ESbox, then this document does not apply.)

If you want to use your own VM image, ensure it has SSH and Samba support. For SSH the UseDNS flag should be turned off, and to support Samba, you may need an often-running cron job to resynchronize the clock with the host, to avoid timestamp drift issues when building. See the Clock Synchronization section.

Networking setup

A very important aspect of the ESbox and VM configuration is the networking mode. Modern VM engines usually allow three modes for how the VM virtualizes network access: host, NAT, and bridged.

• Host is useless for ESbox, since it implies that the outside world cannot see the VM, and ESbox would be on the outside. Don't use this mode.
• NAT is a mode where the VM shares the IP address of the host.

  This is useful in a lot of cases with ESbox, but it is not appropriate in the default configuration recommended by ESbox, where the address 127.0.0.1:2222 is used to talk to the VM.

  1. Shared folders cannot be exposed from the VM to the host, which makes it difficult to use C/C++ indexing and searches with the Maemo SDK. This is because we cannot use port forwarding to remap the Windows CIFS ports to distinguish the host's from the VM's.
  2. SBRSH does not work, since the device communicates with sbrsh running under Scratchbox, but the IP address is the same as the host, which does not run the server. Some complex port forwarding could make this work too, but it's probably not worth it.

  VMware does provide a "vmnet8" interface for NAT which has a unique IP address, but unfortunately, this doesn't work reliably (see this thread for examples).

  If you are able to get this working, then you can use NAT with shared folders and SBRSH.

  But since it's not an easy setup, we don't recommend using this mode, unless you know what you're doing, or unless you have no choice (e.g. QEMU).

• Bridged is a mode where the virtual machine has its own IP address on the LAN.

  For all intents and purposes it appears to be a distinct machine on the network. This gives it enough flexibility to support the shared folder mappings and SBRSH usage mentioned above.

  On the other hand, Bridged mode introduces the inconvenience that the DHCP-assigned address of the VM may change between restarts of the machine. This means you need to manually correct the address (e.g. in the Machine Access pane described below). But ESbox provides helpful dialogs reminding you that you may need to update this address and I've used it myself for months successfully.

So, Bridged mode is the best of the three options and we recommend this one. This is the default for the "server image" provided by the maemovmware project.  If you want to change to NAT, you need to reconfigure the networking devices for the given image in the VM engine of choice. See your documentation for details.

ESbox Build Machine Configuration

• Configuration is under Window > Preferences > ESbox > Build Machines. If you navigate here before configuring anything, or try to create a project and launch first, you will get a message directing you to the page.

  

• You need to select a virtual machine engine (or "Manual configuration") and adjust the settings on all the pages accordingly. Each engine has product-specific configuration.
• Any selected build machine has a unique configuration in Machine Access (the user name, password, target/host addresses and ports) and Shared Folders (the shares providing a mapping between the host and target filesystems) tabs. All machines except for Manually Launched have a VM-specific configuration tab as well.

Shared Folders

On the Shared Folders tab, specify the shared folders that ESbox uses to map filesystem paths between the host and VM filesystem.

You specify here mappings for the existing and configured shared folders in the host and VM. You must manually configure file sharing yourself.

See this document for an overview and setup instructions.

NOTE: ESbox only supports Samba (Windows/CIFS) shared folders currently.

NOTE #2: To expose shares from the VM to the host, you should use Bridged networking in the VM. Otherwise you need to understand a lot more about port forwarding than I do. :)

Configuration

• The default shares are suggestions for the most common setup, so that projects can be built for multiple Scratchbox targets, and so indexing can work. See this document for details.
• If Local share? is yes, you need to configure file sharing on your host and expose a share which can be mounted by the VM. There should be at least This typically should be yes for normal usage (since the Eclipse workspace must be on the local machine). If the setting is no, then in the Maemo SDK VM images, the mount is exposed in /etc/samba/smb.conf.
• "Add Host Share..." lets you add an entry from the shared folders exposed on the host. These must be manually configured.
• "Add VM Share..." lets you add an entry from the shared folders exposed from the virtual machine. This requires that the VM be launched and validated before the button is enabled.
• ESbox can automatically mount folders from the host to the virtual machine and from the virtual machine to the host. This dialog appears when ESbox wants to mount a share:

• The username is, by default, the same as the main user account on the given machine, but you may change it if needed depending on the share's permissions.
• The workgroup/domain may not be required unless you are logged into a Windows domain.
• The password will be remembered for future launches of the mount during the current Eclipse session, but it will be discarded if the mount fails.
• To prevent ESbox from trying to mount the share, change the Auto mount? setting to No.

Dynamic shared folder status

The "Check folder status while editing" checkbox will dynamically validate the shared folders against a running VM, to assist with some issues that may be tedious to debug by using "Validate Machines".

To use this:

• Be sure the Machine Access > Target Address and Host Address fields are valid
• Be sure the VM has been launched (using "Launch Machine" or a previous "Validate Machine" invocation)
• Enable the checkbox.
• Then, edit the entries in the table. The status will be updated as you change the fields.

ESbox Virtual Machine Behavior

Launching

Usually, only an explicit user action (like starting a wizard, validating the VM, or viewing ESbox or Maemo preference panels) will require the VM to be running.

ESbox will connect to a running virtual machine that responds to the configured machine engine, target address, and SSH port in the Machine Access settings.

If the specified engine or configured image is not running, then ESbox will prompt you to launch the configured machine:

If the virtual machine is running, but the target can't be contacted (i.e., the SSH address is incorrect), it presents this variant, which indicates the network settings are probably incorrect:

• Check the "Build Machines" preferences to validate that the Machine Access settings match.
• Check "Network Connections" preferences in case you are behind a proxy but the VM isn't. ESbox uses the proxy settings for all network operations.
• Check the Maemo SDK Virtual Image documentation in case you need to finish configuring your VM. This can be a laborious process if you're using NAT networking, as documented elsewhere in this document.

Both dialogs allow you to launch the virtual machine anew or take one of the actions under "Help" (which will cancel the current operation) and let you look for other solutions.

Choose Launch now when you know the machine is not running, and you want ESbox to start it.

Choose It's running if the machine is running and there was a configuration problem that was fixed in the meantime -- e.g., ESbox was trying to contact the VM before the SSH server was started, or you have just corrected the virtual machine's settings (you ran dhclient or switched between NAT and Bridged modes so that the given address works), or you have fixed the Network Proxy settings in the meantime.

If ESbox still cannot detect the machine, it will report a failure. If you think the VM is running but ESbox tells you it "has stopped running", be sure you are using the same exact machine (watch out for multiple copieson your disk, since the full path to the *.vmx file matters).

Choose Cancel, of course, if you do not want to start the virtual machine. This will cancel any operation that requested it.

While the VM is launching, this dialog is shown:

The dialog will remain for a few seconds after boot time. You may expand it in case the machine has booted but the dialog remains:

Closing this dialog will not cancel any operation. It is only informative, to help you look for common problems and solutions while configuring the machine.

Shared folder mappings

Once you have configured mapping that expose the project to the VM, ESbox will automatically select a project location inside a shared folder for you, when you create or import a project:

In fact, if the dialog presents an error about the project location not being visible, this means your shared folder mappings are insufficient, because no valid mapping satisfies the goal. This mapping depends on the mapping constraints for the union of the selected targets.

Once you have established mappings exposing the Maemo SDK to the host, ESbox will use these mappings to populate the C/C++ indexer settings when you create, import, or convert a project:

(If you need to change your shared folder setup, or change other things that invalidate these settings, you can use Project > Index > Reset Paths and Symbols or Project > Index > Add Missing Paths and Symbols to regenerate them from the current mappings.)

Clock synchronization

When you perform C/C++ builds in ESbox, it is essential that the clock settings in the host and the virtual machine match, since they share a Samba filesystem whose timestamps determine whether any work needs to be done. If they mismatch, then either your Make output will be littered with "timestamp drift" messages or your programs will not be rebuilt as expected.

Unfortunately, the virtual machine engine and configuration influence the clock in different ways. (For example, in VMware, the clock may act differently depending on whether the VMware Tools are installed.) You will need to manually configure your system and experiment to make things work.

(NOTE: these steps are in-progress -- provide feedback to esbox-users at garage.maemo.org if you have better ideas ;)

One approach is:

• Log into the virtual machine
• Disable the /etc/cron.minutely/00resetclock script (i.e. comment out the hwclock command). This currently this runs without appropriate options to respect the timezone. But note that the clock may drift (without VM engine tools support).
• Reconfigure the time zone to match the one you use in the host, using sudo dpkg-reconfigure tzdata.
• Edit /etc/default/rcS and set UTC=no.
• Edit /etc/profile and comment out the line: export TZ=Europe/London.
• Reboot.
• Sync the clock. Run "sudo /sbin/hwclock --localtime --hctosys".
• Verify the time, using "date". This should match the current time on the host.
• Now, you should be able to create files in a shared folder and validate their timestamps match those on the host. In my experiments with VMware Fusion on OS X, unfortunately, the timstamps were off by as much as 5 seconds. So this led to some annoying messages if I built several times in a row, but it seems manageable.

PC-Connectivity Interaction

• Using USB, Bluetooth, WLAN ad-hoc networking (static IPs)

  USB, WLAN Ad-Hoc, and Bluetooth connections use static addresses (192.168.*.15). The default network routing configurations for the host mean that they will only be able to directly communicate back and forth with one "machine" at a time.

  In most cases, you should connect the machine to the host, rather than the virtual machine. Eclipse manages most of the SSH traffic with the device, so the host needs to be able to see the device at its static address. The host cannot see network traffic inside the VM.

  Conversely, from the device point of view, the device can only see the machine it is connected to. For cases where the device needs to open a socket from the host, the device must be able to see the host at the host address (192.168.*.14). This includes Python debugging and SBRSH.

  Note: if you are using NAT networking with the virtual machine, the VM will also be able to see the device at the static address. This is the best of both worlds.

  When using SBRSH, you most likely need to use WLAN instead of a static IP configuration. The device must be visible on the network inside the virtual machine and the host machine, since SBRSH runs on under Scratchbox, and the device needs to mount filesystems from the VM and host.

• Using WLAN networking

  WLAN networking is the most flexible option, as long as your virtual machine is configured for Bridged networking, where it has its own DHCP-acquired address. In this model, the host, the VM, and the device are all peers on the LAN. Thus, the device will be visible to the host and the VM, and the device can see the host and the VM.

  Unfortunately, of course, WLAN is very slow and prone to packet loss. It can be more difficult to use when debugging, for example.

Troubleshooting

• Cannot access the machine
  • If ESbox can't guess your network properly, look at the boot screen for the VM or login to the VM (maemo/maemo) and run sudo dhclient to validate the assigned IP address for the VM, if you're not getting a connection. Only in the case of NAT should you enter 127.0.0.1 as the Target address in ESbox. For non-NAT cases, usually the host address is xx.xx.xx.2 of the target address.
  • "ping" is your friend. If you have connection problems, cross-check the address displayed, e.g., in a launch configuration or in the RSE connection, and verify that you can see if from your host or your VM (depending on context).
  • If you're having trouble accessing port forwarding over SSH, then note that VMware's virtual network port mappings apply even when it is not hosting a running VM, so select different target ports for other VM engines.
• Scratchbox not found?
  • If you have first set up a machine and have had some difficulties with configuring it, you may need to poke ESbox to make it detect Scratchbox and targets. Go to Window > Preferences > Maemo > Installed Targets and hit Refresh.
• Shared folder issues
  • See this document.
• Device access issues
  • The device should be connected to the host (not the VM) for most cases, except for SBRSH (which must use WLAN in most cases).
  • Watch out for firewall software thwarting you. It may block all access to your device. Allow traffic to the 192.168.* IP range.
  • If your LAN uses 192.168.{2,3,4}.* for its own DHCP server, and you want to use the static USB, Bluetooth, or WLAN ad-hoc connections, you need to have separate subnets. Reconfigure these static addresses in the PC-Connectivity Manager and in your host configuration.
  • Avoid host-only networking configurations in your VM. Otherwise you will not be able to contact the device or the Internet.
  • SBRSH needs to communicate in three ways: between the host (running Eclipse), the VM (hosting Scratchbox and running the sbrsh client), and the device (running the sbrsh daemon). In most cases, you must use WLAN for this to work, unless you know how to manually configure routing tables to do this over USB or bluetooth or the ad-hoc WLAN connection.
• Virtual machines launching when not using ESbox?
  • Eclipse sometimes triggers builds for all projects in the workspace, which may be a problem if you are using other products in your installation and do not want to be prompted to launch the VM.
  • The easiest solution is to switch the Build Machine to None. (With this setting, you will not be able to use most ESbox actions, since no installed targets except for Remote Connections will be recognized.) An alternative is to use the Manually Launched Machine setting. (If you use ESbox commands, it will fail to connect immediately if no machine responds to the address.)
  • Alternately, a configured build machine can live alongside other products if you avoid allowing Eclipse to build ESbox projects:
    • Close ESbox projects
    • or
    • Disable Project > Build Automatically and
    • Disable Window > Preferences > Run/Debug > Build (if required) Before Launching