1
Package Download & Installation
<p>Confirm that you're running Virtuoso version <code>08.03.3329</code> or later. The latest versions are available for download <a href="https://shop.openlinksw.com/license_generator/virtuoso-download/">here</a>.</p>
1
Package Download & Installation
<p>Download the <a href="http://download3.openlinksw.com/uda/vad-packages/8.3/val_dav.vad">Virtuoso Authentication Layer (VAL) VAD package</a>.</p>
1
Package Download & Installation
<p>Download the <a href="https://shop.openlinksw.com/license_generator/vad/?submitBtn=MakeLinkPage&vadPackage=http%3A%2F%2Fvirtuoso.openlinksw.com%2Fdata%2Fturtle%2Fapplications%2Fapplications.ttl%23PAVad">Personal Assistant VAD package</a>, which comprises both the client OPAL Client (i.e. Personal Assistant Frontend) and Server components.</p>
1
Package Download & Installation
<p>In the <code>[Flags]</code> section of the <code>virtuoso.ini</code> configuration file, set <code>enable_g_in_sec = 1</code>, and restart the Virtuoso server.</p>
1
Package Download & Installation
<p>In the <code>[URIQA]</code> section of the <code>virtuoso.ini</code> configuration file, set <code>DefaultHost</code> to the canonical host/domain name the Virtuoso server will be accessed on.</p>
1
Package Download & Installation
<p>It is usually best to put the VAD packages into the <code>vad/</code> directory, typically found alongside the <code>bin/</code> directory that holds your Virtuoso binary, replacing any older versions of these VADs that may already be there.</p>
<p>Once the packages have been downloaded, they can be installed via the Virtuoso iSQL command-line or Conductor interfaces. You <em>MUST</em> install the <em>VAL</em> package before installing the <em>Personal Assistant</em> package.</p>
<p>Assuming you are using the Conductor interface, drill down to the <strong>System Admin</strong> -> <strong>Packages</strong> tab.</p>
<ul>
<li>If you put the VAD packages in the <code>vad/</code> directory, you should see them in the long list, where you can tick the boxes to the left of both VADs, and click the <strong>Install/Upgrade</strong> button.</li>
<li>If you put the VAD packages elsewhere, you'll need to install them one-by-one. Scroll down to the <strong>Install package</strong> -> <strong>Install from</strong> section, and click the <strong>Choose File</strong> button, to get a file navigator dialog within which you can navigate to your downloaded VAD packages. (Locate the <em>VAL</em> package, first!) Once you've selected one of those VAD packages, click the <strong>Proceed</strong> button:</li>
</ul>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-vad-install-1.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-vad-install-1.png" width="690" height="349"/></a>
<br><br>
<p>Whether installing one-by-one or both-at-once, you'll be presented with a confirmation pane. Review the VAD(s) listed there, and click the <strong>Proceed</strong> button to install the VAD(s):</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-vad-install-2.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-vad-install-2.png" width="690" height="204"/></a>
<br><br>
<p>If installing one-by-one, repeat for the other (i.e., the <em>Personal Assistant</em> package).</p>
<p>The installation is now complete:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-vad-install-3.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-vad-install-3.png" width="690" height="190"/></a>
<br><br>
<p>Additional information about the <em>VAL</em> package is available <a href="http://vos.openlinksw.com/dataspace/owiki/wiki/VOS/ValWhatWhyHow">here</a>. You don't need to digest all that now. The steps in this guide will walk you through relevant aspects of <em>VAL</em> configuration for use with the <em>Personal Assistant</em>.</p>
2
Configuration
<p>The Virtuoso <code>/chat</code>, <code>/chat/api</code>, and <code>/ws/chat</code> virtual directories <strong>must</strong> be exposed via a secure HTTPS port.</p>
<p>This can be done by either:</p>
<ol>
<li>Using a HTTPS reverse proxy server like nginx or other to forward HTTPS request back to the Virtuoso HTTP virtual directories.</li>
<li>Setting up a Virtuoso <a href="https://docs.openlinksw.com/virtuoso/webserverhttpslistenerset/#webserverhttpslistenersetvt">HTTPS listener</a> and either:
<ol>
<li>Create a copy the definitions for the existing <code>/chat</code>, <code>/chat/api</code>, and <code>/ws/chat</code> virtual directories from the HTTP listener definitions, which were created during VAD package installation.</li>
<li>Alternatively, you can use the SQL command templates below, where the <code>{CNAME}</code> placeholder implies the Canonical Name of the machine hosting your Virtuoso instance, to create the required HTTPS Listener virtual directories via the iSQL command-line or Conductor interfaces:</li>
</ol>
</li>
</ol>
<pre>
DB.DBA.VHOST_REMOVE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/chat'
);
DB.DBA.VHOST_DEFINE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/chat',
ppath=>'/DAV/VAD/personal_assistant/',
is_dav=>1,
is_brws=>0,
def_page=>'index.html',
ses_vars=>0,
sec=>'SSL',
auth_opts=>vector ('https_key', '{db-or-filesystem-hosted-tls-key}', 'https_cert', '{db-or-filesystem-hosted-tls-cert}', 'https_extra_chain_certificates', '{db-or-filesystem-hosted-tls-cert-chain.pem}', 'https_verify', '0', 'https_cv_depth', 10),
is_default_host=>0
);
-- Chat Websockets Endpoint
DB.DBA.VHOST_REMOVE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/ws/chat'
);
DB.DBA.VHOST_DEFINE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/ws/chat',
ppath=>'/SOAP/Http/websockets',
is_dav=>0,
is_brws=>0,
soap_user=>'chat',
ses_vars=>0,
auth_opts=>vector ('https_key', '{db-or-filesystem-hosted-tls-key}', 'https_cert', '{db-or-filesystem-hosted-tls-cert}', 'https_extra_chain_certificates', '{db-or-filesystem-hosted-tls-cert-chain.pem}', 'https_verify', '0', 'https_cv_depth', 10),
opts=>vector ('websocket_service_call', 'OAI.DBA.CHAT', 'websocket_service_connect', 'OAI.DBA.CHAT_WS_CONNECT', 'cors', '*', 'cors_allow_headers', '*', 'cors_restricted', 0),
is_default_host=>0
);
-- Chat API Endpoint
DB.DBA.VHOST_REMOVE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/chat/api'
);
DB.DBA.VHOST_DEFINE (
lhost=>':443',
vhost=>'{CNAME}',
lpath=>'/chat/api',
ppath=>'/SOAP/Http',
is_dav=>0,
is_brws=>0,
realm=>'Chat',
auth_fn=>'OAI.DBA.CHAT_WS_AUTH',
soap_user=>'chat',
ses_vars=>0,
auth_opts=>vector ('https_key', '{db-or-filesystem-hosted-tls-key}', 'https_cert', '{db-or-filesystem-hosted-tls-cert}', 'https_extra_chain_certificates', '{db-or-filesystem-hosted-tls-cert-chain.pem}', 'https_verify', '0', 'https_cv_depth', 10),
opts=>vector ('websocket_service_call', 'OAI.DBA.CHAT', 'websocket_service_connect', 'OAI.DBA.CHAT_WS_CONNECT','cors', '*', 'cors_allow_headers', '*', 'cors_restricted', 0),
is_default_host=>0
);
</pre>
2
Configuration
<p>Using the Conductor's WebDAV Explorer, locate <code>/DAV/VAD/personal_assistant/config.js.example</code>. Copy or rename it to <code>config.js</code> in the same folder (ensuring read permissions for owner, group, and other on the resulting <code>config.js</code>), and edit to set the relevant WebSocket (<code>wsServer</code>) & Base URI (<code>httpServer</code>) for the installation in question.</p>
<pre>
var wsServer='wss://{CNAME}:{PORT}/ws/chat';
var httpServer='https://{CNAME}:{PORT}';
</pre>
<p><strong>Note:</strong> The client UI (a lightweight Single Page Application) can be used as a frontend to any other Personal Assistant chat backend; it is not mandatory to use the client UI on the same instance as the backend.</p>
2
Configuration
<h3>API Keys</h3>
<p>By default, users will be prompted upon connection to enter their own OpenAI API key to use the client UI. However, if required or preferred, one can register an OpenAI API key for system-wide access, using the command <code>registry_set ('openai-chat-app-key', '[API KEY]')</code> via the iSQL command-line or Conductor interfaces. <strong>Note:</strong> all usage will then be billed to that OpenAI API key account. You can disable this modality at any time by executing the command <code>registry_remove ('openai-chat-app-key')</code> via the iSQL command-line or Conductor interfaces.</p>
<h3>Virtuoso Authentication Layer (VAL) Configuration</h3>
<h4>ABAC ACLs and Graph Caching</h4>
<p>Use of this application requires the <a href="https://vos.openlinksw.com/owiki/wiki/VOS/ValWhatWhyHow">Virtuoso Multi-Protocol Authentication Layer (VAL)</a> <code>Graph Caching Mode</code> be enabled, which ensures highly scalable fine-grained attributed-based access control processing.</p>
<p>The <code>Graph Caching Mode</code> can be enabled by via the Conductor UI or via the iSQL command-line.</p>
<h5>Conductor UI</h5>
<p>From the Conductor <strong>System Admin - Packages</strong> tab, select the <em>VAL</em> vad package <em>configure</em> link to right which loads the VAL configuration dialog and check the <em>Cache Named Graph ACLs</em> option:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-conductor-ui.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-conductor-ui.png" width="690" height="243"/></a>
<br><br>
<h5>Command Line</h5>
<p>By simply executing the following command using the Virtuoso <code>isql</code> command line tool:</p>
<pre>
__dbf_set('enable_g_in_sec',1)
</pre>
<h5>Virtuoso Configuration File</h5>
<p>Edit the Virtuoso configuration file <code>virtuoso.ini</code> adding the <code>enable_g_in_sec = 1</code> parameter to the <code>[Flags]</code> section ie</p>
<pre>
[Flags]
enable_g_in_sec = 1
...
</pre>
<h4>Access Controls</h4>
<p>In addition to OpenAI API Keys, it is important to protect the Personal Assistant using attribute-based access control lists (ABACLs). This is achieved by creating a generic ACL that requires users to authenticate (using a variety of authentication protocols) prior to application use.</p>
<p>This can be done using either of these two methods:</p>
<ul>
<li>Using the iSQL command-line tool <strong>only</strong> (where the macro substitution commands used are supported), run the following script situated in <code>{your-instance-cname/DAV/VAD/personal_assistant/sql/chat_acl_rules.sql}</code>:</li>
</ul>
<pre>
SELECT VAL.DBA.val_acl_rule_graph(VAL.DBA.get_default_realm ());
SET U{RULES} $LAST[1];
SPARQL
PREFIX acl: <http://www.w3.org/ns/auth/acl#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX oplacl: <http://www.openlinksw.com/ontology/acl#>
PREFIX oplres: <http://www.openlinksw.com/ontology/restrictions#>
WITH <$U{RULES}>
DELETE { <#rulePublicChat> ?p ?o . }
WHERE { <#rulePublicChat> a acl:Authorization ; ?p ?o . }
INSERT
{
<#rulePublicChat>
a acl:Authorization ;
foaf:maker <#dba> ;
oplacl:hasAccessMode oplacl:Read ,
oplacl:Write ;
acl:accessTo <urn:oai:chat> ;
acl:agent <#groupBasicNetID> ;
oplacl:hasRealm oplacl:DefaultRealm ;
oplacl:hasScope oplacl:Query .
}
;
</pre>
<ul>
<li>Using the Conductor <strong>Interactive SQL</strong> interface:
<ol>
<li>Run the following query to determine the <code>ACL rules graph</code> name (<code>{acl-rules-graph-name}</code>) being used by the Virtuoso instance:</li>
</ol>
</li>
</ul>
<pre>
SELECT VAL.DBA.val_acl_rule_graph(VAL.DBA.get_default_realm ());
</pre>
<ol start="2">
<li>Then, in the following ACL rule, replace <code>{acl-rules-graph-name}</code> with the value returned by the previous step, and run the queries below to generate the required ACL rule:</li>
</ol>
<pre>
SPARQL
PREFIX acl: <http://www.w3.org/ns/auth/acl#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX oplacl: <http://www.openlinksw.com/ontology/acl#>
PREFIX oplres: <http://www.openlinksw.com/ontology/restrictions#>
WITH <{acl-rules-graph-name}>
DELETE
{
<#rulePublicChat>
a acl:Authorization ;
foaf:maker <#dba> ;
oplacl:hasAccessMode oplacl:Read ,
oplacl:Sponge ,
oplacl:Write ;
acl:accessTo <urn:oai:chat> ;
acl:agent <#groupBasicNetID> ;
oplacl:hasRealm oplacl:DefaultRealm ;
oplacl:hasScope oplacl:Query .
}
INSERT
{
<#rulePublicChat>
a acl:Authorization ;
foaf:maker <#dba> ;
oplacl:hasAccessMode oplacl:Read ,
oplacl:Sponge ,
oplacl:Write ;
acl:accessTo <urn:oai:chat> ;
acl:agent <#groupBasicNetID> ;
oplacl:hasRealm oplacl:DefaultRealm ;
oplacl:hasScope oplacl:Query .
};
</pre>
<h4>VAL Scopes Configuration</h4>
<ol>
<li>In the <a href="https://vos.openlinksw.com/owiki/wiki/VOS/ValQuickStartGuide#Scopes">Conductor's VAL configure</a> page:
<ol>
<li>Enable the VAL default ACL <code>scopes</code> and <em>save</em> then click on the <em>Setup</em> button of the <em>ACL Configurator</em> to create default ACL rules:</li>
</ol>
</li>
</ol>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-val-config-1.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-val-config-1.png" width="690" height="366"/></a>
<br><br>
<ol start="2">
<li>Enable <em>Solid Client Authentication Library compatibility mode</em>.</li>
</ol>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-val-config-2.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-val-config-2.png" width="690" height="56"/></a>
<br><br>
<ol start="2">
<li>If the HTTPS Listener is setup in Virtuoso <a href="https://community.openlinksw.com/t/configure-virtuoso-authentication-layer-val-endpoint-https-callback-url/4089">Configure the Virtuoso Authentication Layer (VAL) Secure Endpoint</a> to enable Callback URL from HTTPS.</li>
</ol>
2
Configuration
<p>Incorporating web crawls into query solution production pipelines requires the following steps:</p>
<ol>
<li>Grant <code>SPARQL_SPONGE</code> privileges to the SQL account ("demo" by default) associated with the <code>Demo.demo.execute_sql_query</code> stored procedure that handles SPASQL (SPARQL inside SQL) functionality. This is achieved by executing the command <code>GRANT SPARQL_SPONGE TO demo</code>.</li>
<li>Run the attached script through the iSQL command-line interface.</li>
</ol>
<p><a href="upload://stAB8ZDFmUgq4UiGZ8C5mhKwrbw.zip">assistant_sponging_patch.sql.zip</a> (1.1 KB)</p>
<p>Having successfully performed the steps above, you will be able to execute SPASQL queries such as:</p>
<pre>
SELECT X.sample, X.entityTypeID, X.sampleCount
FROM
(
SPARQL
DEFINE get:soft "soft"
SELECT (SAMPLE(?entity) AS ?sample)
(COUNT(*) AS ?sampleCount)
(?entityType As ?entityTypeID)
FROM <https://www.apple.com/shop/buy-ipad/ipad-pro>
WHERE {?entity a ?entityType . FILTER (CONTAINS(STR(?entityType),"schema"))}
GROUP BY ?entityType
ORDER BY DESC (?sampleCount)
) AS X
</pre>
<p><strong>Note:</strong> <code>Demo.demo.execute_sql_query</code> is the default stored procedure for handling SQL and SPARQL query processing. As part of its default settings, it runs under the identity user âÃÂÃÂdemoâÃÂàwhich can be changed to a different user account if you so desire. It is also important to note that sponging is a sandboxed read-write operation scoped to named graphs denoted by the URL of a sponged document i.e., it doesnâÃÂÃÂt have system-wide graph access.</p>
3
Application Usage
<p>The OpenLink Personal Assistant is only accessible on the secure HTTPS port of the hostname <code>https://{CNAME}:{PORT}/chat</code>:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-application-usage-1.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-application-usage-1.png" width="690" height="331"/></a>
<br><br>
<p>Click on the <strong>Login</strong> button to log in and access the personal assistant:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-application-usage-2.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-application-usage-2.png" width="690" height="245"/></a>
<br><br>
<p>Authenticate by choosing one of the authentication service options presented by Virtuoso's multi-protocol authentication interface:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-application-usage-3.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-application-usage-3.png" width="690" height="265"/></a>
<br><br>
<p>Click on the <strong>Authorize</strong> button to authorize the login request:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-application-usage-4.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-application-usage-4.png" width="690" height="335"/></a>
<br><br>
<p>Enter a valid <code>OpenAI API Key</code> in the designated field, to enable communication with the <code>OpenAI API</code>:</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-application-usage-5.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-application-usage-5.png" width="690" height="337"/></a>
<br><br>
<p>Questions can now be posted in the prompt windows for responses by the Assistant!</p>
4
Personal Assistant Configuration
<p>In this mode, the Assistant operates as an alternative OpenAI model sandbox, with the added ability to execute SQL, SPARQL, and SPASQL queries. This functionality is enabled through the invocation of Virtuoso Stored Procedures published for the Assistant's use, using a callback system.</p>
4
Personal Assistant Configuration
<p>In this mode, the Assistant is equipped with behavior tuning information derived from a JSON-based configuration file, guiding its incorporation of relevant product knowledge graphs when responding to prompts. This process of loosely-coupling a Large Language Model (LLM)-based Assistant and a Knowledge Graph is generally known as Retrieval Augmented Generation (RAG).</p>
<br><br>
<a class="guide-img" data-toggle=lightbox href="http://www.openlinksw.com/images/opal-installation/opal-openlink-assistant-session-ui.png"><img src="http://www.openlinksw.com/images/opal-installation/opal-openlink-assistant-session-ui.png" width="690" height="185"/></a>
<br><br>
<p>Here are examples of the aforementioned JSON-based configuration files that offer powerful templates for configuring the behavior of the Assistant:</p>
<ul>
<li><a href="http://linkeddata.uriburner.com/DAV/VAD/personal_assistant/json/virtuoso-support-assistant-config.json">Virtuoso Support Assistant Configuration</a></li>
<li><a href="http://linkeddata.uriburner.com/DAV/VAD/personal_assistant/json/uda-support-assistant-config.json">ODBC & JDBC Support Assistant Configuration</a></li>
</ul>
5
Sample Prompts
<p>What's your external function call list?</p>
5
Sample Prompts
<p>Show me your external function call list without arguments, and for each, show its expected result output format.</p>
5
Sample Prompts
<p>What is the expected results format for execute_sql_query?</p>
5
Sample Prompts
<p>Test execute_sql_query.</p>
5
Sample Prompts
<p>Write and execute a sample SPARQL query.</p>
5
Sample Prompts
<p>Write and execute a sample SPARQL query related to movies by Spike Lee using the DBpedia SPARQL endpoint.</p>
5
Sample Prompts
<p>Write and execute a sample SPARQL query related to music by Bob Marley using the DBpedia SPARQL endpoint.</p>
5
Sample Prompts
<p>Write and execute a sample SPARQL query using the DBpedia SPARQL endpoint with Paris as the focus.</p>
5
Sample Prompts
<p>Write and execute a sample SPARQL query using the DBpedia SPARQL endpoint with Paris and London as the focus.</p>
5
Sample Prompts
<p>Redo, but for music by Bob Marley using the DBpedia SPARQL endpoint.</p>
5
Sample Prompts
<p>How does OpenLink Virtuoso support data security?</p>
5
Sample Prompts
<p>How does OpenLink Virtuoso handle data integration?</p>
5
Sample Prompts
<p>What are the key features of OpenLink Virtuoso?</p>
5
Sample Prompts
<p>Where do I place my Virtuoso license file?</p>
5
Sample Prompts
<p>How Do I Install Virtuoso on macOS?</p>
5
Sample Prompts
<p>How Do I Install Virtuoso on Windows?</p>
6
Additional Information
<p>Visit the <a href="https://community.openlinksw.com">OpenLink Community Forum</a> for discussions, tips, and more detailed guides.</p>