Looking ahead to SYSTEMS & COMPONENTS 2025, it is clear that increasing connectivity and automation in the off-highway sector is influencing the interaction between man and machine: the "ideal" human-machine interface (HMI) supports the driver and provides them with the right information at the right time. Camera and radar systems, such as those presented in Hanover from 9 to 15 November, facilitate all-round visibility and can increase safety. Augmented reality (XR) is also finding its way into assistance systems and HMI solutions for mobile machinery.

"Mobile machinery must be able to work reliably in a harsh and constantly changing environment. They perform tasks that require the operator's utmost attention," says Petra Kaiser from the DLG (German Agricultural Society). For Kaiser, autonomous functions are a great support for operators. "They make work much easier, while at the same time the machine operator retains control," says Kaiser, who has been Brand Manager at SYSTEMS & COMPONENTS since December 2024 - the leading industry get-together for the international supplier industry for the off-highway sector, which this year has the guiding theme of "Touch smart Efficiency". The wide variety of different machines in the agricultural, forestry and construction industries leads to a high demand for modular and flexible solutions. "This means that new functions for the autonomous operation of a machine can be added piece by piece - a development that is also reflected at the exhibition centre in Hanover," says Kaiser, giving an outlook on the content of the B2B marketplace, which will once again take place as part of the world's leading trade fair Agritechnica.

Reliable under harsh conditions

If the trends in the off-highway sector are transferred to the system architecture of a mobile machine, this means that more and more data from an increasing number of sensors must be processed. Ultimately, this information should be presented to the operator in an intuitive way. Smart and effective HMI technologies are essential for this. They keep the driver up to date with the current situation at work. "A new type of communication between man and machine is required, in which the vehicle also increasingly takes on an active role," explains Kaiser. In this context, effective means that the vehicle helps the driver to understand the situation, ensures safety and provides all the necessary information without overburdening them. HMI solutions, which take centre stage at SYSTEMS & COMPONENTS, provide continuous feedback on important operating data such as speed, fuel consumption, crop yield and machine status. This makes it possible to make informed decisions in the cockpit and work more efficiently.

What's more, mobile machinery is increasingly being controlled remotely. In these cases, it is a particular challenge for human operators to maintain an overview of the machine and its surroundings. The latest developments are therefore focussing on extended reality (XR) technologies. The digital extension of the human sensory organs is intended to take interaction with the machine to a new level. For Christiana Seethaler, Vice President Product Development at TTControl, the integration of XR technologies into HMI products is the "next step towards creating smarter, more reliable off-highway machines." The principle behind it: XR technologies show the machine status in the driver's immediate field of vision, haptic feedback can send tangible signals, for example via a joystick. In addition, acoustic signals draw attention to hazards.

Making working machines "tangible" for people

In combination with external sensors, such XR tools can help to identify obstacles and dangerous environmental factors. This can not only improve safety, but also increase efficiency through predictive working. However: "XR technology must be intuitive enough to enable low-threshold use," explains Martijn Rooker, Innovation Projects & Funding Manager at TTControl. "That's why we are applying a human-centred, scenario-based co-design approach in the ^TheiaXR project. This means that we incorporate experiences from real end users and industrial companies to ensure that all solutions developed are as intuitive and user-friendly as possible," says Rooker, who is coordinating the European research project.

The aim is to make the "invisible" visible and to digitally enhance the perception of operators in the cockpit without restricting their performance. The consortium consists of eleven partners, including TU Dresden, TU Graz, the University of Luxembourg and the Finnish research institute VTT on the scientific side. ^TheiaXR is currently being tested in three scenarios in which mobile machines are used: Slope preparation, logistics and construction. Together with the partners, the aim is to find a solution to visualise gas pipes and power cables using virtual reality, as these are repeatedly destroyed during construction work. The technology could be used to display information on the depth and route of such lines in the machine operator's field of vision, for example by projecting it onto the cockpit windscreen.

Sensor fusion ensures greater safety

Powerful environmental sensor technology not only provides an important building block for the automation of mobile machinery - it also plays a key role in making it safer to use. As a result, SYSTEMS & COMPONENTS will be focussing on solutions for accident prevention. The technology suppliers are confronted with extremely different environmental conditions. Strong contrasts, backlighting and twilight pose major challenges for camera systems. This is one reason why several sensor designs such as LiDAR, ultrasound or radar are used in combination with RGB or infrared cameras. An electronic fusion of the sensor data combines the advantages of the different sensor principles and thus increases the reliability of the assistance system. This allows work cycles to be carried out faster, more effectively and more safely, which ultimately saves time and money.

Modern ultrasonic systems with up to twelve sensors enable a 360-degree all-round view. Depending on the configuration, the sensors only measure the pure distance or also offer object localisation. An algorithm uses triangulation to determine the position of the object in the detection area. Safety certification in accordance with ISO 25119 enables this type of solution to be integrated into a higher-level system for environment detection, which must fulfil the functional safety requirements for agricultural machinery and tractors. This includes, for example, approach controls or emergency braking functions for slow-moving machines.

Collision warning for more precise manoeuvring

One example of the electronic fusion of sensor data and the resulting increased performance of assistance systems is the Collision Avoidance System (CAS) from Bosch Rexroth. Based on precise environment detection using radar, ultrasound and smart cameras, it intuitively warns the operators of excavators, wheel loaders and other machines of impending collisions by vibrating the joystick. The type and intensity of the haptic feedback provides information about the distance to the object. If the machine is operated with two joysticks, the system can communicate the direction from which the object is approaching by vibrating the respective joystick.

In contrast to purely visual or acoustic feedback, attention remains almost fully focussed on the work process, which saves important reaction time. Even with loud ambient noise and without looking at the display, the warning is delivered directly to the driver's hands. For higher automation levels, there is also the option of automatic brake intervention via an electrohydraulically controlled power brake valve - if there is no timely response to the warning, the required brake pressure is automatically built up.

Assistance functions for greater productivity

Haptic collision protection is just one example of how modular assistance systems increase safety in the working environment. While this type of information is currently still mainly provided by the cameras and sensors mounted on the machine itself, in future data generated by the surrounding area, for example by other vehicles, robots and drones, will also be summarised in an intuitive situational picture for the operator. The "Off Highway Twins 2" (OHT-2) project, which is based in the Rhineland region, is taking up this approach. By fusing geodata from the cloud with sensor and telemetry data using modelling, sensor data fusion and AI, comprehensive and detailed models of infrastructure objects and their surroundings, known as off-highway twins, are to be derived in real time. For example, it would be possible to read out how much material an excavator has moved from the movement and drive data and note directly in the terrain model where the excavation has taken place.

"The conceivable scenarios and the added value of such information are key to increasing productivity in various areas of application in agriculture, forestry and construction," emphasises Petra Kaiser. All of this shows: "Autonomous technologies and assistance systems, such as those on show in Hanover from 9 to 15 November, score points through improved processes, more precise work and increased safety. Work processes in the off-highway environment can be precisely planned and coordinated," says the SYSTEMS & COMPONENTS Brand Manager.