Fraunhofer IISB at the Workshop on Reinforcement Learning 2026, University of Mannheim

Last Friday, our team had the privilege of attending and presenting a poster at the Workshop on Reinforcement Learning 2026 at the University of Mannheim.

This work introduces a graph-based RL agent that addresses a persistent bottleneck in power electronics: the slow and complex process of optimizing power module parasitic layouts.

Designing a power module is a delicate balancing act. The goal is to minimize parasitic inductance (for better electrical performance) while keeping maximum junction temperature low (for reliability). Traditionally, this is a time-consuming, manual, iterative task. Although recent approaches automate it using genetic algorithms, each new layout requires training a new genetic algorithm from scratch—there is no knowledge transfer.

Our graph-based PPO agent can design low-temperature and low-parasitic-inductance half-bridge layouts in far fewer steps than genetic algorithms require—even for unseen templates.

Here’s a look at the key breakthroughs we shared in the poster:

1. Problem Formulation

We formulate power module optimization as optimizing the edges of constraint graphs derived from a given template (see Fig. 1). This graph-based view naturally fits a graph-based RL agent that can generalize across different templates.

A template is split into three graphs:

Template Graph — captures the general structure (component placement, connections, etc.).
Horizontal Constraint Graph (HCG)
Vertical Constraint Graph (VCG)

The HCG and VCG are built from the template graph and define the widths and lengths of components in the topology. By modifying edge weights in these constraint graphs, new layouts with varying dimensions are generated.

Fig. 1: Problem Formulation

2. Processing the Graphs

To enable generalization, the three graphs are processed using Graph Neural Networks (GNNs). Fig. 2 illustrates how each graph is encoded.

Fig. 2: How the graphs are processed using GNNs

3. Graph-Based PPO Agent

The node and graph embeddings from the GNNs feed into the Actor and Critic networks of our PPO agent (see Fig. 3).

The Critic receives a concatenation of the three graph embeddings, passed through an MLP to produce a value estimate.
The Actor iterates over all edges in the HCG and VCG. For each edge (between nodes i and j), it concatenates the relevant node embeddings + the template graph embedding, then feeds them into an MLP to output an action for that constraint edge.

*Fig. 3: PPO’s Critic and Actor Networks *

4. Episodic Procedure

The agent is trained on a dataset of power module templates for 10 million steps. The general episodic procedure is shown in Fig. 4 and works as follows:

Sample a template from the dataset and generate its three graphs.
For each step t in the episode:
- Adjust each constraint edge weight by up to ±10%.
- Compute the resulting change in parasitic inductance and temperature (based on the logarithmic layout change from the previous step).
- Assign a reward as a linear combination of the two objectives.

Fig. 4: Environment's episodic logic

5. Results

The generalized RL agent—trained on a dataset of power module templates—was tested on unseen templates. It achieves quality comparable to NSGA-II (a strong genetic algorithm baseline) using 50× fewer steps.

Fig. 5: Layouts found by the RL agent (in 100 steps), NSGA-II, and random sampling (in 5000 steps) for two example templates.

6. Conclusion

This graph-based formulation and RL approach enables training a generalized agent that optimizes even unseen templates 50× faster than NSGA-II—saving expensive, time-consuming electro-thermal simulations.

The work paves the way for more efficient and powerful AI-based methods in power module design—a direction we are actively pursuing at Fraunhofer IISB. Stay tuned for updates!

2026 Workshop on Reinforcement Learning (University of Mannheim)

Authors