<\/p>\n
What we did<\/h2>\n
We developed an application that allows anyone in the organization to query data in a simple and autonomous way. The process begins with the selection of the table on which a query is to be made.<\/p>\n
Next, the person must choose whether to perform the query in SQL or through an internal client tool that operates as a REST API, which uses a specific syntax to build queries and access the data. Then, the language model (LLM) in charge of transforming the natural language query into a structured query is selected. Available options include models such as GPT-3.5 Turbo, GPT-4o, Gemini Pro or Claude 2.<\/strong><\/p>\n Then, the person writes the question in natural language (for example: \u201cWhere does Sof\u00eda Mart\u00ednez Delgado work?\u201d).<\/p>\n Once this configuration is completed, different processes are activated in the backend. First, all contextual information related to the selected table (table and column names and descriptions, data types, etc.) is retrieved from a vectorstore. With this information, and taking into account the selected tool and model, a call to an API (FastAPI) that integrates the available LLMs is launched.<\/p>\n The model receives a complete prompt including:<\/p>\n The model returns the query already generated, either in SQL format or in JSON, depending on the tool chosen.<\/p>\n Before executing the query, the person can review and, if desired, modify the generated query. Once validated, the query process is launched to the database through an API request.<\/p>\n Finally, the requested information is displayed on the screen in response to the query.<\/p>\n <\/p>\n During the development of the project, one of the main challenges was the low quality of the data and the poor documentation of the tables and views, fundamental elements for the language model (LLM) to correctly build the queries.<\/p>\n We encountered the absence of a clear standard in the data. For example, in some tables, string type column values could appear in uppercase, lowercase, with or without accents, and even with incomplete or inconsistent proper names. This created a major problem: even if the model correctly generated the query structure, if the values used for filtering did not exactly match those in the database, the API did not return any results.<\/p>\n To mitigate this situation, we implemented different strategies.<\/p>\n We tried to enrich the model prompt by including, next to the table description, a sample of typical values per column (data profiling). This helped the LLM to better understand how the real data were represented. However, this approach had an important limitation: the considerable increase in the number of tokens, which impacted costs and compatibility with certain models.<\/p>\n To avoid the token limit problem, we designed a solution based on the generation of more tolerant filters using regular expressions (regexp), which would allow ignoring differences in uppercase, lowercase and accents. For example:<\/p>\n This expression makes it possible to retrieve records with spelling or capitalization variations<\/strong> in the value \u201cSof\u00eda\u201d, thus increasing the robustness of the query.<\/p>\n However, this solution also presented challenges: the complexity of the expressions meant that the prompts required very high precision from the model. We had to spend time fine-tuning the instructions and training the prompts well to get the model to consistently generate efficient queries.<\/p>\n Finally, after several iterations, we were able to achieve a satisfactory accuracy rate, where in most cases the model generated valid and effective queries, even in contexts with inconsistent or poorly documented data.<\/p>\n <\/p>\n <\/p>\n <\/p>\n The ability to convert natural language queries into SQL statements or REST API calls represents a significant change in the way organizations access and manage their data. This solution removes technical barriers, allowing non-technical profiles to interact autonomously with information.<\/p>\n Thanks to this tool, companies can streamline their processes, reduce dependence on specialized teams for repetitive tasks and foster a more collaborative and data-driven environment. In an increasingly competitive business environment, having agile and reliable access to information translates into better decisions and greater operational efficiency.<\/p>\n <\/p>\n Do you want to harness the power of LLMs to address your information democratization challenges? We have a team of experts in Generative AI and Data<\/a> who have developed GenIA Ecosystem<\/strong>, an ecosystem of proprietary AI solutions to meet any challenge. Tell us about your project!<\/p>\n Contact with our Data & AI team<\/p>\n <\/div>\n <\/p> ![\"\"](\"https:\/\/www.hiberus.com\/crecemos-contigo\/wp-content\/uploads\/2025\/06\/AI-SQL-soluciones.png\")
<\/p>\n\n
![\"\"](\"https:\/\/www.hiberus.com\/crecemos-contigo\/wp-content\/uploads\/2025\/06\/sql-IA-image.png\")
<\/p>\nProject Challenges<\/h2>\n
Strategy 1: example of values (data profiling)<\/h3>\n
Strategy 2: normalization with regular expressions<\/h3>\n
fullName REGEXP '.*[Ss][Oo\u00d3\u00f3][Ff][Ii\u00cd\u00ed][Aa\u00c1\u00e1].*'<\/span><\/code><\/div>\n<\/div>\n<\/div>\n<\/div>\nBeneficits<\/h2>\n
\n
![\"\"](\"https:\/\/www.hiberus.com\/crecemos-contigo\/wp-content\/uploads\/2025\/06\/sql-IA-image3.png\")
<\/p>\nProject results<\/h2>\n
Model performance<\/h3>\n
\n
Technical performance<\/h3>\n
\n
User experience<\/h3>\n
\n
Operational performance<\/h3>\n
\n
Testing and evolution<\/h3>\n
\n
Want to learn more about our Artificial Intelligence services for SQL queries?<\/h3>\n
<\/ul><\/div>\n