Abstract
Electricity markets are complex environments, which have been suffering continuous transformations due to the increase of renewable based generation and the introduction of new players in the system. In this context, players are forced to re-think their behavior and learn how to act in this dynamic environment in order to get as much benefit as possible from market negotiations. This paper introduces a new learning model to enable players identifying the expected prices of future bilateral agreements, as a way to improve the decision-making process in deciding the opponent players to approach for actual negotiations. The proposed model introduces a con-textual dimension in the well-known Q-Learning algorithm, and includes a simulated annealing process to accelerate the convergence process. The proposed model is integrated in a multi-agent decision support system for electricity market players negotiations, enabling the experimentation of results using real data from the Iberian electricity market.

Abstract
Due to the increment of the energy consumption and dependency of the nowadays lifestyle to the electrical appliances, the essential role of an energy management system in the buildings is realized more than ever. With this motivation, predicting energy consumption is very relevant to support the energy management in buildings. In this paper, the use of an energy management system supported by forecasting models applied to energy consumption prediction is demonstrated. The real-time automatic forecasting system is running separately but integrated with the existing SCADA system. Nine different forecasting approaches to obtain the most reliable estimated energy consumption of the building during the following hours are implemented.

Abstract
The current energy scenario requires actions towards the reduction of energy consumption and the use of renewable resources. In this context, a microgrid is a self-sustained network that can operate connected to the smart grid or in isolation. The long-term scheduling of on/off cycles of devices is a critical problem that has been commonly addressed by centralized approaches. In this work, we propose a novel agent-based method to solve the long-term scheduling problem as a distributed constraint optimization problem (DCOP) by modelling future system configurations rather than reacting to changes. Moreover, with respect to approaches based on decentralised reinforcement learning, we can directly encode system-wide hard constraints (such as for example the Kirchhoff law) which are not easy to represent in a factored representation of the problem. We compare different multi-agent DCOP algorithms showing that the proposed method can find optimal/near-optimal solutions for a specific case study.

Abstract
This paper presents the application of collaborative reinforcement learning models to enable the distributed learning of energy contracts negotiation strategies. The learning model combines the learning process on the best negotiation strategies to apply against each opponent, in each context, from multiple learning sources. The diverse learning sources are the learning processes of several agents, which learn the same problem under different perspectives. By combining the different independent learning processes, it is possible to gather the diverse knowledge and reach a final decision on the most suitable negotiation strategy to be applied. The reinforcement learning process is based on the application of the Q-Learning algorithm; and the continuous combination of the different learning results applies and compares several collaborative learning algorithms, namely BEST-Q, Average (AVE)-Q; Particle Swarm Optimization (PSO)-Q, and Weighted Strategy Sharing (WSS)-Q. Results show that the collaborative learning process enables players’ to correctly identify the negotiation strategy to apply in each moment, context and against each opponent. © Springer Nature Switzerland AG 2019.

Abstract
A key challenge in the power and energy field is the development of decision-support systems that enable studying big problems as a whole. The interoperability between multi-agent systems that address specific parts of the global problem is essential. Ontologies ease the interoperability between heterogeneous systems providing semantic meaning to the information exchanged between the various parties. The use of ontologies within Smart Grids has been proposed based on the Common Information Model, which defines a common vocabulary describing the basic components used in electricity transportation and distribution. However, these ontologies are focused on utilities' needs. The development of ontologies that allow the representation of diverse knowledge sources is essential, aiming at supporting the interaction between entities of different natures, facilitating the interoperability between these systems. This paper proposes a set of ontologies to enable the interoperability between different types of agent-based simulators, namely regarding electricity markets, the smart grid, and residential energy management. A case study based on real data shows the advantages of the proposed approach in enabling comprehensive power system simulation studies.

Abstract
This paper proposes an adaptation of the Q-Learning reinforcement learning algorithm, for the identification of the most probable scenario that a player may face, under different contexts, when negotiating bilateral contracts. For that purpose, the proposed methodology is integrated in a Decision Support System that is capable to generate several different scenarios for each negotiation context. With this complement, the tool can also identify the most probable scenario for the identified negotiation context. A realistic case study is conducted, based on real contracts data, which confirms the learning capabilities of the proposed methodology. It is possible to identify the most probable scenario for each context over the learned period. Nonetheless, the identified scenario might not always be the real negotiation scenario, given the variable nature of such negotiations. However, this work greatly reduces the frequency of such unexpected scenarios, contributing to a greater success of the supported player over time. © 2019, Springer Nature Switzerland AG.