GoldenGate 23ai introduces a powerful new feature called Data Streams, which leverages the AsyncAPI specification and you publish your trail file directly from GoldenGate(and you can specify the CloudEvents format), according to the documentation:
Oracle GoldenGate Data Streams utilizes the AsyncAPI specification for defining asynchronous APIs. This approach enables applications to efficiently subscribe to data streams using a Publish or Subscribe model.
It was hard to me to understand the concept behind this but with the release of GoldenGate 23ai I had some time to play with it and here are my considerations.
Producer
My producer is an extract capturing data from my 19c database, and here you don’t need anything special:
After your extract is up and running, go to Distribution Service > Data Streams and create a new one selecting your trail file
You can also apply some filters
Now that your Data Stream is up, the AsyncApi shows its power, all the specs are there to understand what you can do and what are the format available , the Data Stream reads the trail and publish it (using wss protocol).
In my case I’ve created some “endpoints”, each one to expose an specific operation type:
Consumer/Subscribe
For the Consumer side I’m using the websocat, after the download you can run it from the command line with this syntax:
./websocat.x86_64 wss://USER:'PASSWORD'@SERVER:9102/services/v2/stream/DATA_STREAM_NAMEIn my case I had to use the -k parameter to ignore the certificate:
./websocat.x86_64 wss://USER:'PASSWORD'@SERVER:9102/services/v2/stream/myExample?begin=earliest -k
Now my application can subscribe to the endpoints.
If I would like to consume only the inserts:
./websocat.x86_64 wss://USER:'PASSWORD'@SERVER:9102/services/v2/stream/dsINSERT?begin=earliest -k -v -S | jq -r
And if i need to get only the updates
./websocat.x86_64 wss://USER:'PASSWORD'@SERVER:9102/services/v2/stream/dsUPDATE?begin=earliest -k -v -S | jq -r
Conclusion
If your goal was to share your data in json format you can move from this:
To this:
As usual, there is no silver bullet, each project has specific details, but with this new option, your application/consumer/subscriber can go closer to the producer to get the data.