Introduction to Web Services

Hydstra provides tools to enable access to Hydstra data via SOAP and REST Web Services.

Through the implementation of custom code (to initiate a data request and process the data response) you have direct access to the data you may require for data modelling, data reports or new web based data presentation.

Custom code for SOAP may be developed on any software platform that supports SOAP.

A Perl script is provided as a web service gateway to data access methods provided via the Hydstra DLL (HYDLLP).

The Hydstra Perl webservice URL will be something like http://203.3.195.115/cgi/webservice.server.pl

The related WSDL (Web Services Description Language) file URL will be http://203.3.195.115/cgi/webservice.wsdl

Please contact KISTERS if you require assistance with web services scripting.

REST Request and Response

The data request is made by passing a JSON structure appended as a URL query. For example:

http://realtimedata.water.nsw.gov.au/cgi/webservice.server.pl?{"params":{"site_list":"201001","start_time":"19750801000000","varfrom":"100.00","interval":"day","varto":"100.00","datasource":"A","end_time":"19750802000000","data_type":"mean","multiplier":"1"},"function":"get_ts_traces","version":"2"}

The details of the request JSON structure will depend upon which DLL JSONCall function you choose. The example uses the get_ts_traces DLL function. Please see the HYDLLP documentation for the latest available function names and parameters.

The response to the previous request may look something like this:

{"buff_required": 613, "buff_supplied": 50000, "error_num": 0, "_return": {"traces": [{"trace": [{"q": 130, "v": "0.367", "t": 19750801000000}, {"q": 130, "v": "0.365", "t": 19750802000000}], "site": "201001", "varfrom_details": {"subdesc": "", "variable": "100.00", "short_name": "Level (Metres)", "name": "Stream Water Level", "units": "Metres"}, "site_details": {"short_name": "OXLEY @ EUNGELLA", "name": "OXLEY RIVER AT EUNGELLA"}, "compressed": "0", "quality_codes": {"zzzzzz_130": "Not quality coded - subject to change"}, "varto_details": {"subdesc": "", "variable": "100.00", "short_name": "Level (Metres)", "name": "Stream Water Level", "units": "Metres"}}]}}

The REST web service may also be used for Hydstra systems with multiple DAT paths. The web portal must be correctly configured with a set of web userclasses with matching login credentials in the PASSWD table (the users' PASSWD:WEBUSERCLS fields must be populated).

To enable the web service to access the correct data you must supply the username and password of the required userclass user as part of the web service URL.

http://mydomain.com/cgi/webservice.server.pl?{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array"}}&user=myname&pass=mypassword

SOAP Request and Response

The data request is a block of XML which includes a JSON data structure specifying:

• a Hydstra DLL function name

• the function's parameter names and values

• the version number of the DLL data request protocol

plus the buffer size for the data response (in bytes).

Please see the HYDLLP documentation for the latest available function names and parameters.

For example, use the following to request the mean daily stream level archive data for sites HYDSYS01 and HYDSYS02 from 1/8/1975 to 1/9/1975:

The data response is a block of XML containing a JSON data structure including:

• a process error code (and message if necessary)

• the size of the response (bytes)

• various relevant meta data plus the requested data types and values

• the size of the supplied buffer size (bytes)

The corresponding response XML is:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
               xmlns:xsd="http://www.w3.org/2001/XMLSchema"
               soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
    <JSonCallResponse xmlns="http://203.3.195.115/Hydstra">
      <s-gensym3 xsi:type="xsd:string">
        {"error_num":0,
        "buff_required":"3477",
        "return":{
          "traces":[{
            "site_details":{
              "short_name":"Hydstra Test Station",
              "name":"Hydstra Test Station - Composite data"},
            "quality_codes":{
              "1":"Good continuous records"},
            "trace":[
              {"v":"0.363","t":19750801000000,"q":1},
              {"v":"0.368","t":19750802000000,"q":1},
              {"v":"0.363","t":19750803000000,"q":1},
              {"v":"0.464","t":19750804000000,"q":1},
              {"v":"0.505","t":19750805000000,"q":1},
              {"v":"0.417","t":19750806000000,"q":1},
              {"v":"0.368","t":19750807000000,"q":1},
              {"v":"0.318","t":19750808000000,"q":1},
              {"v":"0.294","t":19750809000000,"q":1},
              {"v":"0.285","t":19750810000000,"q":1},
              {"v":"0.308","t":19750811000000,"q":1},
              {"v":"0.461","t":19750812000000,"q":1},
              {"v":"0.507","t":19750813000000,"q":1},
              {"v":"0.457","t":19750814000000,"q":1},
              {"v":"0.396","t":19750815000000,"q":1},
              {"v":"0.369","t":19750816000000,"q":1},
              {"v":"0.354","t":19750817000000,"q":1},
              {"v":"0.344","t":19750818000000,"q":1},
              {"v":"0.328","t":19750819000000,"q":1},
              {"v":"0.275","t":19750820000000,"q":1},
              {"v":"0.251","t":19750821000000,"q":1},
              {"v":"0.243","t":19750822000000,"q":1},
              {"v":"0.237","t":19750823000000,"q":1},
              {"v":"0.228","t":19750824000000,"q":1},
              {"v":"0.222","t":19750825000000,"q":1},
              {"v":"0.221","t":19750826000000,"q":1},
              {"v":"0.267","t":19750827000000,"q":1},
              {"v":"0.349","t":19750828000000,"q":1},
              {"v":"0.300","t":19750829000000,"q":1},
              {"v":"0.254","t":19750830000000,"q":1},
              {"v":"0.231","t":19750831000000,"q":1},
              {"v":"0.220","t":19750901000000,"q":1}],
            "varfrom_details":{
              "short_name":"Level     (Metres)",
              "subdesc":"",
              "variable":" 100.00",
              "units":"Metres","name":"Stream Water Level"},
            "site":"HYDSYS01",
            "varto_details":{
              "short_name":"Level     (Metres)",
              "subdesc":"",
              "variable":" 100.00",
              "units":"Metres",
              "name":"Stream Water Level"}
            },
            {"site_details":{
               "short_name":"Hydstra Test Station",
               "name":"Hydstra Test Station - Secondary rainfall data"        },
             "quality_codes":{
               "1":"Good continuous records"},
             "trace":[
               {"v":"0.363","t":19750801000000,"q":1},
               {"v":"0.368","t":19750802000000,"q":1},
               {"v":"0.363","t":19750803000000,"q":1},
               {"v":"0.464","t":19750804000000,"q":1},
               {"v":"0.505","t":19750805000000,"q":1},
               {"v":"0.417","t":19750806000000,"q":1},
               {"v":"0.368","t":19750807000000,"q":1},
               {"v":"0.318","t":19750808000000,"q":1},
               {"v":"0.294","t":19750809000000,"q":1},
               {"v":"0.285","t":19750810000000,"q":1},
               {"v":"0.308","t":19750811000000,"q":1},
               {"v":"0.461","t":19750812000000,"q":1},
               {"v":"0.507","t":19750813000000,"q":1},
               {"v":"0.457","t":19750814000000,"q":1},
               {"v":"0.396","t":19750815000000,"q":1},
               {"v":"0.369","t":19750816000000,"q":1},
               {"v":"0.354","t":19750817000000,"q":1},
               {"v":"0.344","t":19750818000000,"q":1},
               {"v":"0.328","t":19750819000000,"q":1},
               {"v":"0.275","t":19750820000000,"q":1},
               {"v":"0.251","t":19750821000000,"q":1},
               {"v":"0.243","t":19750822000000,"q":1},
               {"v":"0.237","t":19750823000000,"q":1},
               {"v":"0.228","t":19750824000000,"q":1},
               {"v":"0.222","t":19750825000000,"q":1},
               {"v":"0.221","t":19750826000000,"q":1},
               {"v":"0.267","t":19750827000000,"q":1},
               {"v":"0.349","t":19750828000000,"q":1},
               {"v":"0.300","t":19750829000000,"q":1},
               {"v":"0.254","t":19750830000000,"q":1},
               {"v":"0.231","t":19750831000000,"q":1},
               {"v":"0.220","t":19750901000000,"q":1}],
             "varfrom_details":{
               "short_name":"Level     (Metres)",
               "subdesc":"",
               "variable":" 100.00",
               "units":"Metres",
               "name":"Stream Water Level"},
             "site":"HYDSYS02",
             "varto_details":{
               "short_name":"Level     (Metres)",
               "subdesc":"",
               "variable":" 100.00",
               "units":"Metres",
               "name":"Stream Water Level"}
            }
          ]
        },
        "buff_supplied":8192}
      </s-gensym3>
    </JSonCallResponse>
</soap:Body>
</soap:Envelope>

The SOAP web service may also be used for Hydstra systems with multiple DAT paths. The web portal must be correctly configured with a set of web userclasses with matching login credentials in the PASSWD table (the users' PASSWD:WEBUSERCLS fields must be populated).

To enable the web service to access the correct data, you must supply the username and password of the required userclass user as part of the SOAP request. There are several ways to do this in Perl (see ..\HYD\SYS\perl\examples\webservice.client.pl):

    my $buffersize = 1000000;
    my %parameters = (
      'function' => 'get_ts_traces',
      'version' => 2,
      'params'   => {
        'site_list' => $siteid,
        'datasource' => 'A',
        'varfrom'    => '100.00',
        'varto'      => '100.00',
        'start_time' => $start,
        'end_time'   => $end,
        'data_type' => 'mean',
        'interval'   => 'day',
        'multiplier' => '1'
      },
      'user'   => $username,
      'pass' => $password,
    );
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (proxy request Perl reference)\n", HashDump(\%parameters), "\n");
    my $json = SOAP::Lite
                -> uri($uri)
                -> proxy($proxy, options => {compress_threshold => 10000})
                -> JSonCall(\%parameters,$buffersize)
                -> result;
    Prt('-S', "returned JSON:\n", $json);

    # -----------------------------------------------------
    my $request_string = qq[{"function":"get_ts_traces",
      "version":2,
      "params":{
        "site_list":"$siteid",
        "datasource":"A",
        "varfrom":"100.00",
        "varto":"100.00",
        "start_time":"$start",
        "end_time":"$end",
        "data_type":"mean",
        "interval":"day",
        "multiplier":"1"
      },
      "user":"$username",
      "pass":"$password",
    }];
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (proxy request string)\n$request_string\n");
    $json = SOAP::Lite
            -> uri($uri)
            -> proxy($proxy, options => {compress_threshold => 10000})
            -> JSonCall($request_string,$buffersize)
            -> result;
    Prt('-S', "returned JSON:\n", $json);

Sample VBA Excel code

Sample provided at ..\HYD\SYS\perl\examples\webservice.bas

Attribute VB_Name = "service"
Option Explicit

Private Const c_domain As String = "203.3.195.115"

Sub helloWorld()
    Dim sURL As String
    Dim sEnv As String
    Dim xmlhttp As New MSXML2.XMLHTTP26
    Dim xmlDoc As New DOMDocument
    Dim oNodeList As IXMLDOMSelection
    Dim n As Long

    sURL = "http://" & c_domain & "/cgi/webservice.server.pl"

    sEnv = "<?xml version=""1.0"" encoding=""UTF-8""?>"
    sEnv = sEnv & "<soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"""
    sEnv = sEnv & " xmlns:soapenc=""http://schemas.xmlsoap.org/soap/encoding/"""
    sEnv = sEnv & " xmlns:xsd = ""http://www.w3.org/2001/XMLSchema"""
    sEnv = sEnv & " soap:encodingStyle=""http://schemas.xmlsoap.org/soap/encoding/"""
    sEnv = sEnv & " xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">"
    sEnv = sEnv & "<soap:Body>"
    sEnv = sEnv & "<helloWorld xmlns=""http://" & c_domain & "/Hydstra"" xsi:nil=""true"" />"
    sEnv = sEnv & "</soap:Body>"
    sEnv = sEnv & "</soap:Envelope>"

    With xmlhttp
     .Open "post", sURL, False
     .setRequestHeader "Content-Type", "text/xml; charset=utf-8"
     .setRequestHeader "soapAction", "http://" & c_domain & "/Hydstra#helloWorld"
     .setRequestHeader "Accept-encoding", "deflate"
    .send sEnv
     xmlDoc.LoadXml .responseText
     Set oNodeList = xmlDoc.selectNodes("soap:Envelope/soap:Body/helloWorldResponse")
     MsgBox c_domain & " helloWorld response" & vbCrLf & oNodeList.Item(0).Text

    End With
End Sub

Sub JSonCall()
Dim sURL As String
Dim sEnv As String
Dim xmlhttp As New MSXML2.XMLHTTP26
Dim xmlDoc As New DOMDocument
Dim oNodeList As IXMLDOMSelection
Dim JSonContent As String

sURL = "http://" & c_domain & "/cgi/webservice.server.pl"

sEnv = "<?xml version=""1.0"" encoding=""UTF-8""?>"
sEnv = sEnv & "<soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"""
sEnv = sEnv & " xmlns:soapenc=""http://schemas.xmlsoap.org/soap/encoding/"""
sEnv = sEnv & " xmlns:xsd = ""http://www.w3.org/2001/XMLSchema"""
sEnv = sEnv & " soap:encodingStyle=""http://schemas.xmlsoap.org/soap/encoding/"""
sEnv = sEnv & " xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">"
sEnv = sEnv & "<soap:Body>"
sEnv = sEnv & "<JSonCall xmlns=""http://" & c_domain & "/Hydstra"">"
sEnv = sEnv & "<c-gensym4 xsi:type=""xsd:string"">"
sEnv = sEnv & " {"
sEnv = sEnv & " ""params"": {"
sEnv = sEnv & " ""start_time"": ""19750501000000"","
sEnv = sEnv & " ""varfrom"": ""100.00"","
sEnv = sEnv & " ""interval"": ""day"","
sEnv = sEnv & " ""varto"": ""100.00"","
sEnv = sEnv & " ""datasource"": ""A"","
sEnv = sEnv & " ""end_time"": ""19750510000000"","
sEnv = sEnv & " ""data_type"": ""mean"","
sEnv = sEnv & " ""site_list"": ""HYDSYS01"","
sEnv = sEnv & " ""multiplier"": ""1"""
sEnv = sEnv & " },"
sEnv = sEnv & " ""function"": ""get_ts_traces"","
sEnv = sEnv & " ""version"": 1"
sEnv = sEnv & " }"
sEnv = sEnv & " </c-gensym4>"
sEnv = sEnv & " <c-gensym7 xsi:type=""xsd:int"">1000000</c-gensym7>"
sEnv = sEnv & "</JSonCall>"
sEnv = sEnv & "</soap:Body>"
sEnv = sEnv & "</soap:Envelope>"

With xmlhttp
    .Open "post", sURL, False
    .setRequestHeader "Content-Type", "text/xml; charset=utf-8"
    .setRequestHeader "soapAction", "http://" & c_domain & "/Hydstra#JSonCall"
    .setRequestHeader "Accept-encoding", "deflate"
    .send sEnv

    xmlDoc.LoadXml .responseText
    Set oNodeList = xmlDoc.selectNodes("soap:Envelope/soap:Body/JSonCallResponse")
    JSonContent = oNodeList.Item(0).Text
    MsgBox c_domain & " JSonCall response:" & vbCrLf & JSonContent

End With
End Sub

Sample Perl code for SOAP

A sample Perl script is provided to exercise several variants of SOAP requests at ..\HYD\SYS\perl\examples\webservice.client.pl

use strict;
use Compress::Zlib;
use SOAP::Lite;
#use SOAP::Lite 'trace', 'debug';
use JSON;
use CGI;
require 'hydlib.pl';

sub urldecode {
    my $s = shift;
    $s =~ s/\%([A-Fa-f0-9]{2})/pack('C', hex($1))/seg;
    $s =~ s/\+/ /g;
    return $s;
}

main: {
    system('cls');

    my $domain = 'realtimedata.water.nsw.gov.au';
    my $uri     = "http://$domain/Hydstra";
    my $proxy   = "http://$domain/cgi/webservice.server.pl";
    my $service = "http://$domain/wini/webservice.wsdl";
    my $query   = new CGI;
    print $query->header(-Accept_encoding => 'deflate');

    Prt('-S', NowStr(), " Started $0\n");
    Prt('-S', "domain: $domain\nuri: $uri\nproxy: $proxy\nservice: $service\n\n");

    # -----------------------------------------------------
    Prt('-S', NowStr(), " - running helloWorld (proxy)\n");
    my $return_string = SOAP::Lite
                     -> uri($uri)
                     -> proxy($proxy, options => {compress_threshold => 10000})
                     -> helloWorld
                     -> result;
    Prt('-S', "helloWorld returned [$return_string]\n\n");

    # -----------------------------------------------------
    Prt('-S', NowStr(), " - running helloWorld (wsdl)\n");
    $return_string = SOAP::Lite
                     -> service($service)
                     -> helloWorld;
    Prt('-S', "helloWorld returned [$return_string]\n");

    # -----------------------------------------------------
    my $buffersize = 1000000;
    my %parameters = (
      'function' => 'get_ts_traces',
      'version' => 2,
      'params'   => {
        'site_list' => '201001',
        'datasource' => 'A',
        'varfrom'    => '100.00',
        'varto'      => '100.00',
        'start_time' => '19750501000000',
        'end_time'   => '19750510000000',
        'data_type' => 'mean',
        'interval'   => 'day',
        'multiplier' => '1'
      }
    );
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (proxy request Perl reference)\n", HashDump(\%parameters), "\n");
    my $json = SOAP::Lite
                -> uri($uri)
                -> proxy($proxy, options => {compress_threshold => 10000})
                -> JSonCall(\%parameters,$buffersize)
                -> result;
    Prt('-S', "returned JSON:\n", $json);

    # -----------------------------------------------------
    my $request_string = qq[{"function":"get_ts_traces",
      "version":2,
      "params":{
        "site_list":"201001",
        "datasource":"A",
        "varfrom":"100.00",
        "varto":"100.00",
        "start_time":"19750501000000",
        "end_time":"19750510000000",
        "data_type":"mean",
        "interval":"day",
        "multiplier":"1"
      }
    }];
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (proxy request string)\n$request_string\n");
    $json = SOAP::Lite
            -> uri($uri)
            -> proxy($proxy, options => {compress_threshold => 10000})
            -> JSonCall($request_string,$buffersize)
            -> result;
    Prt('-S', "returned JSON:\n", $json);

    # -----------------------------------------------------
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (wsdl request string)\n$request_string\n");
    $json = SOAP::Lite
             -> service($service)
             -> JSonCall($request_string,$buffersize);
    Prt('-S', "returned JSON:\n", $json);

    # -----------------------------------------------------
    $request_string = qq[%7B%22function%22%3A%22get_ts_traces%22%2C%22version%22%3A2%2C%22params%22%3A%7B%22site_list%22%3A%22201001%22%2C%22datasource%22%3A%22A%22%2C%22varfrom%22%3A%22100.00%22%2C%22varto%22%3A%22100.00%22%2C%22start_time%22%3A%2219750501000000%22%2C%22end_time%22%3A%2219750510000000%22%2C%22data_type%22%3A%22mean%22%2C%22interval%22%3A%22day%22%2C%22multiplier%22%3A%221%22%7D%7D];
    Prt('-S', "\n\n", NowStr(), " - running JSonCall (wsdl encoded request string)\n$request_string\n");
    $json = SOAP::Lite
             -> service($service)
             -> JSonCall($request_string,$buffersize);
    Prt('-S', "returned JSON:\n", $json);

    Prt('-S', "\n\n", NowStr(), " end\n\n");
}

Sample WSDL (Web Services Description Language) file

Sample provided at \HYD\SYS\WDEF\wini\webservice.wsdl

Usually all you will need to amend are the URL domains at installation.

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="HydstraWebService"
             targetNamespace="http://127.0.0.1/Hydstra"
             xmlns="http://schemas.xmlsoap.org/wsdl/"
             xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
             xmlns:tns="http://127.0.0.1/Hydstra"
             xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<documentation>
    This webservice enables SOAP data requests from the Hydstra water database of the [enter your organisation name here]. Please visit http://127.0.0.1/cgi/webservice.server.pl for detailed documentation.
</documentation>

<message name="helloWorldRequest"/>
<message name="helloWorldResponse">
    <part name="response" type="xsd:string"/>
</message>

<message name="JSonCallRequest">
    <part name="call" type="xsd:string"/>
</message>
<message name="JSonCallResponse">
    <part name="response" type="xsd:string"/>
</message>

<portType name="PortType">
    <operation name="helloWorld">
      <input message="tns:helloWorldRequest"/>
      <output message="tns:helloWorldResponse"/>
    </operation>
    <operation name="JSonCall">
      <input message="tns:JSonCallRequest"/>
      <output message="tns:JSonCallResponse"/>
    </operation>
</portType>

<binding name="Hydstra_Binding" type="tns:PortType">
    <soap:binding style="rpc"
                  transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="helloWorld" namespace="http://127.0.0.1/Hydstra">
      <soap:operation soapAction="http://127.0.0.1/Hydstra#helloWorld"/>
     <input>
        <soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
                   namespace="http://127.0.0.1/Hydstra"
                   use="encoded"/>
      </input>
      <output>
        <soap:body namespace="http://127.0.0.1/Hydstra"
                   encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
                   use="encoded"/>
      </output>
    </operation>

    <operation name="JSonCall" namespace="http://127.0.0.1/Hydstra">
      <soap:operation soapAction="http://127.0.0.1/Hydstra#JSonCall"/>
      <input>
        <soap:body namespace="http://127.0.0.1/Hydstra"
                   encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
                   use="encoded"/>
      </input>
      <output>
       <soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
                   namespace="http://127.0.0.1/Hydstra"
                   use="encoded"/>
      </output>
    </operation>
</binding>

<service name="HydstraWebService">
    <documentation>
      This webservice enables SOAP data requests from the Hydstra water database of the [enter your organisation name here]. Please visit http://127.0.0.1/cgi/webservice.server.pl for detailed documentation.
    </documentation>
    <port binding="tns:Hydstra_Binding" name="Hydstra_Port">
      <soap:address location="http://127.0.0.1/cgi/webservice.server.pl"/>
    </port>
</service>
</definitions>