Bitwacker Associates

ChatBS, the Context-aware LLM Exploratory Sandbox uses the OpenAI Completion API service (GPT and Llama family models) to answer questions. Each sentence in a ChatBS result is automatically linked to a Google query to facilate fact-checking. If requested, ChatBS can then use the OpenAI API to construct an entity/relation graph of these results in the form [‘entity1’, ‘relationship’, ‘entity2’]. ChatBS then uses entity linking to look up both the entities and relationships against Wikidata entities and properties, constructing a JSON-LD graph as is proceeds. For each re-submission of the user’s prompt we display the LLM response and the resulting RDF (if requested)

ISWC 2024 demo abstract: The recent widespread public availability of generative large language models (LLMs) has drawn much attention from the academic community to run experiments in order to learn more about their strengths and drawbacks. From prompt engineering and fine-tuning to fact-checking and task-solving, researchers have pursued several approaches to try to take advantage of these tools. As some of the most powerful LLMs are “closed” and only accessible through web APIs with prior authorization, combining LLMs with the open web is still a challenge. In this evolving landscape, tools that can facilitate the exploration of the capabilities and limitations of LLMs are desirable, especially when connecting with traditional web features such as search and structured data. This article presents ChatBS, a web-based exploratory sandbox for LLMs, working as a front-end for prompting LLMs with user inputs. It provides features such as entity resolution from open knowledge graphs, web search using LLM outputs, as well as popular prompting techniques (e.g. multiple submissions, “step-by-step”). ChatBS has been extensively used in Rensselaer Polytechnic Institute’s Data INCITE courses and research, serving as key tool for utilizing LLMs outputs at scale in these contexts.

ChatBS Live App: https://inciteprojects.idea.rpi.edu/chatbs/app/chatbs/

ChatBS Project Page: https://tw.rpi.edu/project/chatbs-context-aware-llm-exploratory-sandbox