Add support to monitor energy inside virtual machines

README.md

@@ -14,6 +14,7 @@ Detailed documentation (including user and reference guides) are available at: [
 - Monitor power consumption of each method at runtime
 - Uses a Java agent, no source code instrumentation needed
 - Uses Intel RAPL (powercap interface) for getting accurate power reading on GNU/Linux, our research-based regression models on Raspberry Pi devices, and a custom program monitor (using a RAPL driver) for accurate power readings on Windows
+- Monitor energy of Java applications running in virtual machines
 - Provides real-time power consumption of every method in the monitored program
 - Provides total energy for every method on program exit
 
@@ -25,7 +26,7 @@ To build JoularJX, you need Java 11+ and Maven, then just build:
 mvn clean install -DskipTests
 ```
 
-Alternatively, you can use the Maven wrappen shipped with the project with the command:
+Alternatively, you can use the Maven wrapper shipped with the project with the command:
 
 ```
 Linux: ./mvnw clean install -DskipTests
@@ -34,7 +35,7 @@ Windows: ./mvnw.cmd clean install -DskipTests
 
 JoularJX depend on the following software or packages in order to get power reading:
 - On Windows, JoularJX uses a custom power monitor program that uses the [Windows RAPL driver by Hubblo](https://github.com/hubblo-org/windows-rapl-driver), and therefore require installing the driver first, and runs on Intel or AMD CPUs (since Ryzen).
-- On Windows, to read the data from the RAPL driver, we use a custom program monitor called [Power Monitor for Windows](https://github.com/joular/WinPowerMonitor). It used to be part of JoularJX, but now is in its own repository. Download the binary (or compile the source code), and specify its path in ```config.properties```.
+- On Windows, to read the data from the RAPL driver, we use a custom program monitor called [Power Monitor for Windows](https://github.com/joular/WinPowerMonitor). It used to be part of JoularJX, but now it is in its own repository. Download the binary (or compile the source code), and specify its path in ```config.properties```.
 - On PC/server GNU/Linux, JoularJX uses Intel RAPL interface through powercap, and therefore requires running on an Intel CPU or an AMD CPU (since Ryzen).
 - On macOS, JoularJX uses `powermetrics`, a tool bundled with macOS which requires running with `sudo` access. It is recommended to authorize the current users to run `/usr/bin/powermetrics` without requiring a password by making the proper modification to the `sudoers` file.
 - On Raspberry Pi devices on GNU/Linux, JoularJX uses our own research-based regression models to estimate CPU power consumption with support for the following device models (we support all revisions of each model lineup. However, the model is generated and trained on a specific revision, listed between brackets, and the accuracy is best on this particular revision):
@@ -46,7 +47,8 @@ JoularJX depend on the following software or packages in order to get power read
   - Model 3 B+ (rev 1.3), for 32-bit OS
   - Model 4 B (rev 1.1, and rev 1.2), for both 32 bits and 64-bit OS
   - Model 400 (rev 1.0), for 64-bit OS
-  - Model 5 B (rev 1.0), for 64 bits OS
+  - Model 5 B (rev 1.0), for 64-bit OS
+- On virtual machines, JoularJX reads the power consumption of the virtual machine (measured in the host) from a file shared between the host and the guest.
 
 We also support Asus Tinker Board (S).
 
@@ -98,10 +100,17 @@ JoularJX can be configured by modifying the ```config.properties``` files:
 - ```save-call-trees-runtime-data```: write runtime call trees power consumption in a CSV file. For each monitoring cycle (1 second), a new CSV file will be generated, containing the runtime power consumption of the call trees. The generated files will include timestamps in their names.
 - ```overwrite-call-trees-runtime-data```: overwrite runtime call trees power data file, or if set to false, it will write new file for each monitoring cycle.
 - ```application-server```: properly handles application servers and frameworks (Sprig Boot, Tomcat, etc.). Set ```true``` when running on application servers. If false, the monitoring loop will check if the JVM is destroyed, hence closing JoularJX when the application ends (in regular Java application). If true, JoularJX will continue to monitor correctly as the JVM isn't destroyed in a application server.
+- ```vm-power-path```: the path for the power consumption of the virtual machine. Inside a virtual machine, indicate the file containing power consumption of the VM (which is usually a file in the host that is shared with the guest).
+- ```vm-power-format```: power format of the shared VM power file. We currently support two formats: ```watts``` (a file containing one float value which is the power consumption of the VM), and ```powerjoular``` (a csv file generated by [PowerJoular](https://github.com/joular/powerjoular) in the host, containing 3 columns: timestamp, CPU utilization of the VM and CPU power of the VM).
 
 You can install the jar package (and the PowerMonitor.exe on Windows) wherever you want, and call it in the ```javaagent``` with the full path.
 However, ```config.properties``` must be copied to the same folder as where you run the Java command.
 
+In virtual machines, JoularJX requires two steps:
+- Installing a power monitoring tool in the host machine, which will monitor the virtual machine power consumption every second and writing it to a file (to be shared with the guest VM).
+For example, you can use our [PowerJoular](https://github.com/joular/powerjoular).
+- Use JoularJ in the guest VM while specifying the path of the power file shared with the host and its format.
+
 ## Generated files
 
 For real-time power data or the total energy at the program exit, JoularJX generated two CSV files:

config.properties

@@ -73,4 +73,22 @@ application-server=false
 
 # Path for our power monitor program on Windows
 # On Windows, please escape slashes twice
-powermonitor-path=C:\\joularjx\\PowerMonitor.exe
+powermonitor-path=C:\\joularjx\\PowerMonitor.exe
+
+# Monitoring inside virtual machines
+# Values: true, false
+vm-monitoring=true
+
+# Path for power consumption of the virtual machine
+# Inside a virtual machine, indicate the file containing power consumption of the VM
+# File usually shared with host to propagate the power of the VM from the host
+# And use this value as the emulated CPU power inside the VM
+vm-power-path=/tmp/power.csv
+
+# Power format of the shared VM power file
+# We currently support two formats:
+# powerjoular: a csv file generated by PowerJoular in the host, containing
+#              3 columns: timestamp, CPU utilization of the VM and CPU power of the VM
+# watts: a file containing one float value which is the power consumption of the VM
+# Values: powerjoular, watts
+vm-power-format=watts

pom.xml

@@ -18,7 +18,7 @@
 
     <groupId>org.noureddine</groupId>
     <artifactId>joularjx</artifactId>
-    <version>2.9.0</version>
+    <version>3.0.0</version>
 
     <packaging>jar</packaging>
     <name>${project.artifactId}</name>

src/main/java/org/noureddine/joularjx/Agent.java

@@ -46,7 +46,7 @@ public static void premain(String args, Instrumentation inst) {
         JoularJXLogging.updateLevel(properties.getLoggerLevel());
 
         logger.info("+---------------------------------+");
-        logger.info("| JoularJX Agent Version 2.9.0    |");
+        logger.info("| JoularJX Agent Version 3.0.0    |");
         logger.info("+---------------------------------+");
 
         ThreadMXBean threadBean = createThreadBean();

src/main/java/org/noureddine/joularjx/cpu/Cpu.java

@@ -13,11 +13,11 @@
 
 public interface Cpu extends AutoCloseable {
 
-    void initialize();
+    public void initialize();
 
-    double getInitialPower();
+    public double getInitialPower();
 
-    double getCurrentPower(double cpuLoad);
+    public double getCurrentPower(double cpuLoad);
 
-    double getMaxPower(double cpuLoad);
+    public double getMaxPower(double cpuLoad);
 }

src/main/java/org/noureddine/joularjx/cpu/CpuFactory.java

@@ -41,6 +41,11 @@ public static Cpu getCpu(final AgentProperties properties) {
         String osArch = System.getProperty("os.arch").toLowerCase();
         logger.info("Initializing for platform: '" + osName + "' running on architecture: '" + osArch + '\'');
 
+        if (properties.isVirtualMachine()) {
+            logger.info("Initializing for running inside a virtual machine");
+            return new VirtualMachine(properties.getVMPowerPath(), properties.getVMPowerFormat());
+        }
+
         if (osName.contains("win")) {
             return new IntelWindows(properties.getPowerMonitorPath());
         }

src/main/java/org/noureddine/joularjx/cpu/IntelWindows.java

@@ -16,7 +16,6 @@
 
 import java.io.BufferedReader;
 import java.io.InputStreamReader;
-import java.util.Objects;
 import java.util.logging.Level;
 import java.util.logging.Logger;
 

src/main/java/org/noureddine/joularjx/cpu/RaspberryPi.java

@@ -207,6 +207,7 @@ public void close() {
     /**
      * Nothing to do here. Method only useful for RAPL
      */
+    @Override
     public double getMaxPower(final double cpuLoad) {
         return 0;
     }

src/main/java/org/noureddine/joularjx/cpu/VirtualMachine.java

@@ -0,0 +1,115 @@
+/*
+ * Copyright (c) 2021-2024, Adel Noureddine, Université de Pau et des Pays de l'Adour.
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the
+ * GNU General Public License v3.0 only (GPL-3.0-only)
+ * which accompanies this distribution, and is available at
+ * https://www.gnu.org/licenses/gpl-3.0.en.html
+ *
+ * Author : Adel Noureddine
+ */
+
+package org.noureddine.joularjx.cpu;
+
+import org.noureddine.joularjx.utils.JoularJXLogging;
+import java.util.logging.Logger;
+import java.nio.file.Files;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.FileSystems;
+import java.nio.file.FileSystem;
+import java.util.logging.Level;
+
+public class VirtualMachine implements Cpu {
+
+    private static final Logger logger = JoularJXLogging.getLogger();
+
+    private static String VM_POWER_PATH_NAME;
+
+    private static String VM_POWER_FORMAT;
+
+    private Path VM_POWER_PATH;
+
+    private final FileSystem fileSystem;
+
+    public VirtualMachine(String VMPowerPath, String VMPowerFormat) {
+        this(FileSystems.getDefault());
+        VM_POWER_PATH_NAME = VMPowerPath;
+        VM_POWER_FORMAT = VMPowerFormat;
+    }
+
+    public VirtualMachine(final FileSystem fileSystem) {
+        this.fileSystem = fileSystem;
+    }
+
+    @Override
+    public void initialize() {
+        // Check if VM_POWER_PATH exists and can be read
+        this.VM_POWER_PATH = fileSystem.getPath(VM_POWER_PATH_NAME);
+
+        if (Files.exists(this.VM_POWER_PATH)) {
+            checkFileReadable(this.VM_POWER_PATH);
+        } else {
+            logger.log(Level.SEVERE, "The shared VM power file cannot be found. Exiting...");
+            System.exit(1);
+        }
+    }
+
+    /**
+     * Check that the passed file can be read by the program. Log error message and exit if reading the file is not
+     * possible.
+     * @param file the file to check the read access
+     */
+    private void checkFileReadable(final Path file) {
+        if (!Files.isReadable(file)) {
+            logger.log(Level.SEVERE, "Failed to read the shared VM power file. Please check you have permissions to read it.");
+            System.exit(1);
+        }
+    }
+
+    /**
+     * The power is approximated based on the CPU load, so it does not need an offset.
+     *
+     * @return 0
+     */
+    @Override
+    public double getInitialPower() {
+        return 0;
+    }
+
+    @Override
+    public double getCurrentPower(double cpuLoad) {
+        double powerData = 0.0;
+
+
+        try {
+            if (VM_POWER_FORMAT.equals("watts")) {
+                powerData = Double.parseDouble(Files.readString(VM_POWER_PATH));
+            } else if (VM_POWER_FORMAT.equals("powerjoular")) {
+                String[] powerDataInfo = Files.readString(VM_POWER_PATH).split(",");
+                // Get 3rd column (index 2) for power consumption
+                powerData = Double.parseDouble(powerDataInfo[2]);
+            } else {
+                logger.log(Level.WARNING, "Power data format for VM not supported. Returning 0.");
+            }
+        } catch (IOException exception) {
+            logger.throwing(getClass().getName(), "getCurrentPower", exception);
+        }
+
+        return powerData;
+    }
+
+    /**
+     * Nothing to do here. Method only useful for RAPL
+     */
+    @Override
+    public double getMaxPower(double cpuLoad) {
+        return 0;
+    }
+
+    @Override
+    public void close() {
+        // Nothing to do for virtual machines
+    }
+
+}

src/main/java/org/noureddine/joularjx/monitor/MonitoringHandler.java

@@ -14,7 +14,6 @@
 import java.lang.management.ThreadMXBean;
 import java.util.ArrayList;
 import java.util.HashMap;
-import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
 import java.util.Map.Entry;

src/main/java/org/noureddine/joularjx/utils/AgentProperties.java

@@ -42,6 +42,9 @@ public class AgentProperties {
     private static final String OVERWRITE_CT_RUNTIME_DATA_PROPERTY = "overwrite-call-trees-runtime-data";
     private static final String STACK_MONITORING_SAMPLE_RATE_PROPERTY = "stack-monitoring-sample-rate";
     private static final String APPLICATION_SERVER_PROPERTY = "application-server";
+    private static final String VM_MONITORING_PROPERTY = "vm-monitoring";
+    public static final String VM_POWER_PATH_PROPERTY = "vm-power-path";
+    private static final String VM_POWER_FORMAT_PROPERTY = "vm-power-format";
 
     /**
      * Loaded configuration properties
@@ -60,6 +63,9 @@ public class AgentProperties {
     private final boolean overwriteCtRuntimeData;
     private final int stackMonitoringSampleRate;
     private final boolean applicationServer;
+    private final boolean vmMonitoring;
+    private final String vmPowerPath;
+    private final String vmPowerFormat;
 
     /**
      * Instantiate a new instance which will load the properties
@@ -79,6 +85,9 @@ public AgentProperties(FileSystem fileSystem) {
         this.overwriteCtRuntimeData = loadOverwriteCallTreeRuntimeData();
         this.stackMonitoringSampleRate = loadStackMonitoringSampleRate();
         this.applicationServer = loadApplicationServer();
+        this.vmMonitoring = loadVMMonitoring();
+        this.vmPowerPath = loadVMPowerPath();
+        this.vmPowerFormat = loadVMPowerFormat();
     }
 
     public AgentProperties() {
@@ -132,6 +141,12 @@ public boolean saveCallTreesRuntimeData() {
 
     public boolean isApplicationServer() { return this.applicationServer; }
 
+    public boolean isVirtualMachine() { return this.vmMonitoring; }
+
+    public String getVMPowerPath() { return this.vmPowerPath; }
+
+    public String getVMPowerFormat() { return this.vmPowerFormat; }
+
     private Properties loadProperties(FileSystem fileSystem) {
         Properties result = new Properties();
 
@@ -226,4 +241,16 @@ private Optional<Path> getPropertiesPathIfExists(FileSystem fileSystem) {
 
         return Optional.of(path);
     }
+
+    public boolean loadVMMonitoring() {
+        return Boolean.parseBoolean(properties.getProperty(VM_MONITORING_PROPERTY));
+    }
+
+    public String loadVMPowerPath() {
+        return properties.getProperty(VM_POWER_PATH_PROPERTY);
+    }
+
+    public String loadVMPowerFormat() {
+        return properties.getProperty(VM_POWER_FORMAT_PROPERTY);
+    }
 }

src/test/java/org/noureddine/joularjx/cpu/RaplLinuxTest.java

@@ -69,7 +69,7 @@ void psysFileSupported() throws IOException {
         assertEquals(1.0, cpu.getCurrentPower(0));
     }
 
-    @Test
+   @Test
     void pkgFileSupported() throws IOException {
         Path pkg = fileSystem.getPath(RaplLinux.RAPL_PKG);
         Path pkgMax = fileSystem.getPath(RaplLinux.RAPL_PKG_MAX);
@@ -103,7 +103,7 @@ void pkgAndDramFileSupported() throws IOException {
         assertEquals(2.0, cpu.getCurrentPower(0));
     }
 
-    @Test
+@Test
     @ExpectSystemExitWithStatus(1)
     void raplFileNotReadable() throws IOException {
         Path psys = fileSystem.getPath(RaplLinux.RAPL_PSYS);