It will summarise documents, advise users of the mCitizen app and search for the information they need – among other capabilities of  PLLuM, a large language model developed by a consortium led by Wrocław University of Science and Technology. Everyone can use the Polish researchers' solution.

PLLuM (Polish Large Language Model), a large language model project based on artificial intelligence technology, allows texts to be processed and generated in Polish and will support the development of digital competencies and innovation in public administration and business.

A solution to satisfy Polish needs

Assoc. Prof. Tomasz Kajdanowicz, PhD, DSc, Eng, head of the Department of Artificial Intelligence at the Faculty of Computer Science and Telecommunications, doesn’t doubt that it was necessary to create and develop the Polish model. “We can use other programs today, but the question is whether we want the Polish economy, administration, and science to be entirely dependent on foreign solutions?” asks the Wrocław Tech researcher.

Also, he points out that the currently available foreign models are not neutral, and their development is driven by the interests of non-Polish companies that don’t share information about their products, which can lead to a filtering of the content they provide with no inclusion of the Polish cultural and historical perspective.

“There are already studies that show this. The long-term development of AI in Poland, with PLLuM being far from a one-off project, will entail a lot of investment in future technological solutions and Polish experts and scientists working in the field,” emphasises Prof. Kajdanowicz of W4.

Almost a year of hard work

The efforts to make PLLuM a reality started in 2023 as an initiative of researchers from Wrocław University of Science and Technology, who were already involved in language technology research as part of the CLARIN-PL project. It resulted in outcomes such as research infrastructure used mainly in the humanities and social sciences. Over the past five years, researchers have been working on a variety of databases and natural language processing tools.

Ultimately, in addition to scientists from Wrocław University of Science and Technology, specialists from the Institute of Computer Science of the Polish Academy of Sciences, the Institute of Slavic Studies of the Polish Academy of Sciences, the Scientific and Academic Computer Network, the Information Processing Centre and the University of Łódź contributed to the creation of PLLuM. The Ministry of Digitalisation has funnelled over 14 m PLN into the project.

“It took us eleven months to build a whole family of language models for natural language processing with many applications, including but not limited to information extraction and limited content understanding,” says project coordinator Assoc. Prof. Maciej Piasecki, PhD, DSc, Eng. of Wrocław Tech’s Department of Artificial Intelligence.

“We have also created a special version of the model that can answer questions based on any document database. It has already been tested by the Ministry of Digitalisation, providing information on things related to public institutions and especially what is available through the mCitizen application. The solution has achieved very high efficiency,” he adds.

The chatbot model has already been made available in a browser-based form at http://pllum.clarin-pl.eu/, where anyone can test it.

Available in multiple versions, the Polish language model allows flexible and scalable customisation – using between 8 and 70 billion parameters. Also, it precisely generates content in Polish. Smaller versions work well for quick tasks, while larger models offer higher precision and contextual consistency in understanding Polish.

“It’s important to emphasise that we didn’t build PLLuM to compete with existing language models, but as our own, Polish engine, over which we have full control, knowing exactly what it contains. In addition, we have full control over the process of building, fine-tuning, and implementing it,  which means that our public institutions and companies can be confident of the security of their data. While working on the solution, we also gained extensive knowledge that will certainly pay off in the future,” points out Prof. Piasecki.

Exceptional transparency

The versions intended for commercial use apply textual resources from owners who have licensed the consortium, as well as resources that can be used to build a fully open model. The scientific models also make use of publicly available datasets, such as Common Crawl.

“Our model is highly transparent in legal terms, which makes it unique. I don't know of another solution like this. We didn’t take any shortcuts when creating it – we did a lot of legal analyses and cooperated with the Ministry of Justice, so we’re sure that we use all the data in accordance with the applicable law,” emphasises Prof. Piasecki.