diff --git a/coherence-demo/explore-clients/explore-clients.md b/coherence-demo/explore-clients/explore-clients.md new file mode 100644 index 0000000..5211cbf --- /dev/null +++ b/coherence-demo/explore-clients/explore-clients.md @@ -0,0 +1,147 @@ +# Explore Python, JavaScript and Go Clients + +## Introduction + +This lab walks you through accessing Coherence from Python, JavaScript and Go clients. + +Coherence provides the ability for clients in various languages to connect to a cluster +using [gRPC](https://grpc.io/) as the network transport. + +For Java clients, connecting using gRPC provides an alternative to **Coherence*Extend** connections and can be advantageous when you need to connect through a load balancer as gRPC uses HTTP/2 under the covers and is more load balancer friendly. + +If you want to connect to Coherence from JavaScript, Python, or Go clients, then gRPC is the only protocol supported. + +For any of the language options, from a cluster perspective, you must include the **`coherence-grpc-proxy`** module, with which the server-side gRPC proxy will accept the gRPC connections and carry out work on behalf of the clients. + +> Note: You can inspect the source code for each of the clients in the last lab. + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Use the Python client to monitor prices +* Use the JavaScript client to add trades +* Use the Go client to issue a stock-split + +### Prerequisites + +* You should have completed the previous labs. + +## Task 1: Monitor Prices Using Python + +> Note: Ensure you have enabled price updated in the primary cluster. + +1. Open a new terminal by using **`File`** -> **`New Tab`** in the existing terminal. + + ![New Terminal](images/new-terminal.png "New Terminal") + +2. Change to the following directory: + ```bash + cd coherence-demo/clients/py + ``` + +3. Run the python command to monitor prices: + ```bash + python3.8 main.py monitor + ``` + + Because prices updates are still on via the dashboard, you should see the updates prices being displayed similar to below: + + ```bash + 2024-11-25 09:08:49,674 - coherence - INFO - Session [ce32beb1-604e-49d3-bf00-b6d3b7f827a7] connected to [localhost:1408]. + Listening for price changes. Press CTRL-C to finish. + Price changed for NFLX, new=$20.13, old=$20.11, change=$0.02 + Price changed for NFLX, new=$19.53, old=$20.13, change=$-0.60 + Price changed for GOOG, new=$21.09, old=$21.13, change=$-0.04 + Price changed for AAPL, new=$18.59, old=$19.30, change=$-0.71 + ``` + +## Task 2: Add Stocks using the JavaScript + +1. Open a new terminal by using **`File`** -> **`New Tab`** in the existing terminal. + +2. Change to the following directory: + ```bash + cd coherence-demo/clients/js + ``` + +3. Issue the following to view the options applicable to the JavaScript client: + ```bash + node main.js + ``` + + You should see output similar to the following: + + ```bash + Usage: main.js command + The following commands are supported: + size - display the cache sizes + monitor - monitor prices + add-trades - add random trades, specify symbol and count + stock-split - stock split, specify symbol and factor + ``` + + > Note: Each of the clients has the same command line arguments which allows us to see how Coherence is accessed from a variety of clients. + +4. Add 5,000 ORCL trades by issuing the following command: + ```bash + node main.js add-trades ORCL 5000 + ``` + + You should see output similar to the following: + + ```bash + 2024-11-25 09:13:49,674: Adding 5000 random trades for ORCL... + 2024-11-25 09:13:50,604: Trades cache size is now 111000 + ``` + + > Note: Confirm in the primary and secondary cluster dashboards that the trade count has increased for the `ORCL` stock. + +## Task 3: Issue a Stock Spit using Go + +1. Click on the checkbox next to **`Real-time Price Updates`** to disable random stock price updates. + +2. Open a new terminal by using **`File`** -> **`New Tab`** in the existing terminal. + +3. Change to the following directory: + ```bash + cd coherence-demo/clients/go + ``` + +4. Create an Executable Go Binary using the following: + ```bash + go build -o go-demo . + ls -l go-demo + ``` + + You should see output similar to the following: + ```bash + -rwxrwxr-x. 1 opc opc 17663730 Nov 25 01:23 go-demo + ``` + +5. Run the following to issue a **3:1** stock split for **`ORCL`** shares which will update the quantity by a factor of 3 and divide the price by a factor of 3. + + ```bash + ./go-demo stock-split ORCL 3 + 2024/11/25 01:24:53 session: aacdcd08-01a8-4954-8bab-76123e45708f connected to address localhost:1408 + 2024/11/25 01:24:55 Updated quantity for 27622 trades + 2024/11/25 01:24:56 Updated price for 27622 trades + 2024/11/25 01:24:56 Updated price for ORCL from $22.21 to $7.40 + ``` + + > Note: If you have time you can explore the three clients and run various commands in each of the clients. + +## Learn More + +* [Developing Remote Clients using gRPC](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.2.0/develop-remote-clients/getting-started-grpc.html) +* [Coherence Python Client](https://github.com/oracle/coherence-py-client) +* [Coherence JavaScript Client](https://github.com/oracle/coherence-js-client) +* [Coherence Go Client](https://github.com/oracle/coherence-go-client) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/explore-clients/images/new-terminal.png b/coherence-demo/explore-clients/images/new-terminal.png new file mode 100644 index 0000000..f2bd573 Binary files /dev/null and b/coherence-demo/explore-clients/images/new-terminal.png differ diff --git a/coherence-demo/explore-code/explore-code.md b/coherence-demo/explore-code/explore-code.md new file mode 100644 index 0000000..088b2bc --- /dev/null +++ b/coherence-demo/explore-code/explore-code.md @@ -0,0 +1,1052 @@ +# Explore the Application Code + +## Introduction + +This lab walks you through the application code allowing you to gain an understanding of how the code works. + +> Note: You can choose any of the languages you wish to inspect. + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Explore Java code +* Explore Python Code +* Explore JavaScript Code +* Explore Go Code + +### Prerequisites + +* You should have completed the previous labs. + +## Task 1: Open the code in the "Files" application + +1. In the remote session, choose **`Activities`** and select the **`Files`** icon to open the **"Files"** application. + + ![Files](images/files.png "Files") + +2. Double click **`coherence-demo`** to change to that directory. + + ![Directory](images/directory.png "Directory") + +## Task 2: Explore the pom.xml, configuration and startup + +1. Application Startup + + The application is started using the following command: + + ```bash + mvn -P grid-edition exec:exec + ``` + + This starts the application using the exec-maven-plugin which sets the following arguments in the project in [pom.xml](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/pom.xml#L351): + + | Argument | Usage | + |-----------------------------------------------------|---------------------------------------------------------------------------------------| + | -Dcoherence.log.level=7 | Set the Coherence log level to 7 to provide more verbose output that the default of 5 | + | -Dcoherence.management=all | Enables management for the cluster | + | -Dcoherence.wka=127.0.0.1 | Scopes the cluster to the localhost only | + | -Dcoherence.ttl=0 | Ensures cluster traffic does not go outside this VM | + | -Dcoherence.grpc.server.port=1408 | Enable the gRPC Proxy on port 1408 | + | -Dcoherence.metrics.http.enabled=${metrics.enabled} | If set to true, enabled Coherence metrics, see the | + + The startup class is [com.oracle.coherence.demo.application.Launcher](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/Launcher.java) which does the following: + + * Determines the time zone and sets sensible primary and secondary cluster names using system properties, as well as Jaegar endpoint + * Sets the following system property which indicates the cache configuration file to load: + + `System.setProperty("coherence.cacheconfig", "cache-config.xml")` + * Calls Coherence.main(args) which is the main entry point for starting coherence + + +2. Configuration Files + + As part of the build, the relevant cache configuration file (cache-config-grid-edition.xml) and override file (tangosol-coherence-override-grid-edition.xml) + are copied from `src/main/resources/` to the target directory. See [here](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/pom.xml#L469). + + **Cache Configuration** + + The cache configuration file (cache-config.xml in our case) defines caches and other services, for the cluster. A few areas of particular interest are: + + 1. An interceptor shown below, or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/src/main/resources/cache-config-grid-edition.xml#L46), + which is run on startup of the Coherence cluster: + ```xml + + + + com.oracle.coherence.demo.application.BootstrapInterceptor + + + + ``` + In this context, interceptors allow us to write code to react to various lifecycle events such as when the cache configuration is activated or disposed. In this case we are running the `BootstrapInterceptor` class, explained below, to boostrap the application. Other interceptors can be created to respond to other events such as member left, partition events and cache mutation events. + 2. Cache Scheme Mapping, shown below or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/src/main/resources/cache-config-grid-edition.xml#L51) + defines the mapping from the cache name to a caching scheme and shows the domain classes for each cache, which are explained further below. + ```xml + + + Trade + federated-scheme + java.lang.String + com.oracle.coherence.demo.model.Trade + + + Price + federated-scheme + java.lang.String + com.oracle.coherence.demo.model.Price + + + ``` + 3. Federated Service Definitions, shown in part below or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/src/main/resources/cache-config-grid-edition.xml#L71) + which defines the actual federated scheme with its read-write backing map to write through to a database (in memory database for convenience in our case). We can also see the topology is defined as `Active`, which is a reference to the topology in the override file below. + ```xml + + federated-scheme + + + + + BINARY + + + 5000 + + + + + com.oracle.coherence.demo.cachestore.JpaCacheStore + + + ... + + + + 2s + + + true + ... + + + Active + + + + ``` + 4. A Http Proxy server which has the JAX-RS resources [ApplicationResourceConfig](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/ApplicationResourceConfig.java) + which serves the static application and [ServiceResourceConfig](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/ServiceResourceConfig.java) which serves various REST endpoints through which the HTML/JavaScript application interacts. + + **Operational Override** + + The tangosol-coherence.xml operational deployment descriptor, or its override, specifies the operational and run-time settings that control clustering, communication, and data management services. + For this demonstration we are configuring federation cluster members in the override file shown below, or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/b632f832fe9860e9eb6fb454f13a4158367d0f23/src/main/resources/tangosol-coherence-override-grid-edition.xml#L38) + or `src/main/resources/tangosol-coherence-override-grid-edition.xml`. + + ```xml + + + + PrimaryCluster + start + + +
127.0.0.1
+ 7574 +
+
+
+ + SecondaryCluster + pause + + +
127.0.0.1
+ 7575 +
+
+
+
+ + + Active + PrimaryCluster + SecondaryCluster + + +
+ ``` + +## Task 2: Explore the Java Code + +1. Domain Classes + + There are two classes that hold the main data for the application: + + * `Price` - holds information regarding symbols and their current price + + Source code: `src/main/java/com/oracle/coherence/demo/model/Price.java` or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/model/Price.java) + + Class definition (excluding getters/ setters, etc) + + ```java + @Entity + @XmlRootElement(name = "price") + @XmlAccessorType(XmlAccessType.PROPERTY) + @PortableType(id = 1003) + public class Price { + + /** + * The symbol (ticker code) of the equity for the {@link Price}. + */ + @Id + private String symbol; + + /** + * The price of the symbol. + */ + private double price; + + ... + } + ``` + + The `Entity` and `Id` annotations are for JPA persistence and the `PortableType` annotation is used at compile time to instrument the class to + enable Portable Object Format serialization which is a fast and compact serialization format used by Coherence. + + * `Trade` - holds individual trades and the price they traded for + + Source Code: `src/main/java/com/oracle/coherence/demo/model/Trade.java` or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/model/Trade.java) + + Class definition (excluding getters/ setters, etc) + + ```java + @Entity + @XmlRootElement(name = "trade") + @XmlAccessorType(XmlAccessType.PROPERTY) + @PortableType(id = 1004) + public class Trade { + /** + * The unique identifier for this trade. + */ + @Id + private String id; + + /** + * The symbol (ticker code) of the equity for the {@link Trade}. + */ + private String symbol; + + /** + * The number of shares for the {@link Trade}. + */ + private int quantity; + + /** + * The price at which the shares in the {@link Trade} were acquired. + */ + private double price; + + ... + } + ``` + +2. Bootstrap Interceptor and Utilities classes + + Available at `src/main/java/com/oracle/coherence/demo/application/BootstrapInterceptor.java` or on [GitHub](src/main/java/com/oracle/coherence/demo/application/BootstrapInterceptor.java). + This class contains code run at startup of the cluster member to determine if it should load the initial data, and then opens the main dashboard. The main part of this code runs the following: + + ```java + if (loadData) { + Utilities.addIndexes(); + Utilities.populatePrices(); + Utilities.createPositions(); + } + ``` + + The `Utilities` class available at `src/main/java/com/oracle/coherence/demo/application/Utilities.java` or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/Utilities.java) + has various methods that the application calls. We have included some relevant snippets below, which extra tracing code removed, for viewing. + + * getPricesCache() + + ```java + public static NamedCache getPricesCache() { + return Coherence.getInstance().getSession().getCache(PRICE_CACHE); + } + ``` + + The above code retrieves the coherence session from the Coherence instance and uses the session to get a reference to a cache. Similar code is used for all the caches. + + * addIndexes() + + ```java + public static void addIndexes() { + NamedCache tradesCache = getTradesCache(); + + tradesCache.addIndex(Trade::getSymbol, true, null); + tradesCache.addIndex(Trade::getPurchaseValue, false, null); + tradesCache.addIndex(Trade::getQuantity, false, null); + + System.out.println(" Done"); + } + ``` + + The above code adds three indexes using value extractors, such as `Trade::getSymbol`, which extract the specified method value + and stores a deserialized version of this for fast access when issuing queries and aggregations. + + * populatePrices() + + ```java + public static void populatePrices() { + NamedCache pricesCaches = getPricesCache(); + for (String symbol : SYMBOLS) { + Price price = new Price(symbol, INITIAL_PRICE); + pricesCaches.put(price.getSymbol(), price); + } + } + ``` + + The above code loops through the `SYMBOLS`, which is an array of stock symbols, and inserts initial prices. + + * createPositions() , which calls `createPositions(null, NR_POSITIONS_TO_CREATE)` + + ```java + public static void createPositions(String symbolToInsert, int count) { + Logger.out(String.format("Creating %d Positions...", count)); + + NamedCache tradesCache = getTradesCache(); + NamedCache priceCache = getPricesCache(); + + boolean singleSymbol = symbolToInsert != null; + + Map localPrices = new HashMap<>(priceCache.getAll(priceCache.keySet())); + HashMap trades = new HashMap<>(); + Random random = ThreadLocalRandom.current(); + + for (int i = 0; i < count; i++) { + // create a random position + String symbol = singleSymbol ? symbolToInsert : SYMBOLS[random.nextInt(SYMBOLS.length)]; + int amount = random.nextInt(1000) + 1; + double price = localPrices.get(symbol).getPrice(); + + Trade trade = new Trade(symbol, amount, price); + + trades.put(trade.getId(), trade); + + // batch the putAll's at 100,000 + if (i % 100_000 == 0) { + Logger.out("Flushing trades from HashMap to Coherence cache..."); + tradesCache.putAll(trades); + trades.clear(); + } + } + + // insert any remaining trades not previously flushed + if (!trades.isEmpty()) { + tradesCache.putAll(trades); + } + Logger.out(String.format("Creation Complete! (Cache contains %d positions) ", tradesCache.size())); + } + ``` + + The above code creates positions for a single symbol, or if `symbol` is null for random symbols. + It utilizes a more efficient `putAll` of a `Map` to quickly add the number of positions required. + +3. Other Classes + + There are various other components to the Java based JAX-RS application. You can explore the various classes and packages below via the explorer or via the direct GitHub links. + + * ChartDataResource.java in `src/main/java/com/oracle/coherence/demo/application` or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/ChartDataResource.java) + + This class contains a JAX-RS endpoint that is called from the HTML/ JavaScript application, that will do aggregation, get prices and invoke code across all + members, to determine the number of Trades each member has. Some relevant code below is: + + ```java + @GET + @Path("{updatePrices}") + @Produces( {APPLICATION_JSON, APPLICATION_XML, TEXT_PLAIN}) + public Response getChartData(@PathParam("updatePrices") boolean updatePrices) { + ... + // retrieve the trade summary using a custom aggregator across the members + Map mapTradesBySymbol = trades.aggregate(GroupAggregator.createInstance(Trade::getSymbol, + new TradeSummaryAggregator())); + + InvocationService invocationService = (InvocationService) CacheFactory.getService("InvocationService"); + + // determine the storage enabled members for the membership query + Set storageEnabledMembers = + ((DistributedCacheService) trades.getCacheService()).getOwnershipEnabledMembers(); + + // determine the member information + Map memberInfoMap = + invocationService.query(new GetMemberInfo(trades.getCacheName()), storageEnabledMembers); + + // establish the chart data + ChartData data = new ChartData(CacheFactory.getCluster().getTimeMillis(), + mapTradesBySymbol, + symbolPrice, + memberInfoMap.values(), + stopWatch.getElapsedTimeIn(TimeUnit.MILLISECONDS)); + } + ``` + + * EventsResource.java in `src/main/java/com/oracle/coherence/demo/application` or on [GitHub](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/java/com/oracle/coherence/demo/application/EventsResource.java) + + This class contains methods to set up a Server Sent Events (SSE) broadcaster and uses Coherence Map Events to send any changes to the SSE channel. + Some relevant methods are shown below: + + ```java + @PostConstruct + void createBroadcaster() { + this.broadcaster = sse.newBroadcaster(); + this.prices = Utilities.getPricesCache(); + + // add a MapListener on the Price cache on startup to push event changes to the event broadcaster + prices.addMapListener(new SimpleMapListener() + .addUpdateHandler(e->broadcaster.broadcast(createEvent("priceUpdate", + e.getNewValue().getSymbol(), e.getOldValue().getPrice(), e.getNewValue().getPrice())))); + } + + private OutboundSseEvent createEvent(String name, String symbol, double oldPrice, double newPrice) { + return sse.newEventBuilder() + .name(name) + .data(Price.class, new PriceUpdate(symbol, oldPrice, newPrice)) + .mediaType(APPLICATION_JSON_TYPE) + .build(); + } + + /** + * Registers an event listener for the specified {@link SseEventSink}. + * + * @param eventSink provided {@link SseEventSink} + */ + @GET + @Path("subscribe") + @Produces(MediaType.SERVER_SENT_EVENTS) + public void registerEventListener(@Context SseEventSink eventSink) { + broadcaster.register(eventSink); + eventSink.send(sse.newEvent("begin", new Date().toString())); + } + ``` + +## Task 3: Explore the Application HTML and JavaScript + +Since the focus of this lab is Coherence, we have not included detailed information about the HTML and Angular components. + +The main components are highlight below: + + | Directory / File | Description | GitHub Link | + |-------------------------------------------------|----------------------------------------|---------------------------------------------------------------------------------------------------------------------------| + | src/main/resource/web | Base directory for the Web application | [Link](https://github.com/coherence-community/coherence-demo/tree/1412/src/main/resources/web) | + | src/main/resources/web/index.html | Main application entry point | [Link](https://github.com/coherence-community/coherence-demo/tree/1412/src/main/resources/web/index.html) | + | src/main/resources/web/javscripts/controller.js | Angular Controller | [Link](https://github.com/coherence-community/coherence-demo/blob/1412/src/main/resources/web/javascripts/controllers.js) | + + +## Task 4: Explore the Python Code + +1. Code and Serialization + + The **Python** code is available in the following location: + + **clients/py/main.py**
+ **[main.py](https://github.com/coherence-community/coherence-demo/blob/1412/clients/py/main.py)** + + For all the clients, by default, data is serialized into JSON and stored as native JSON objects in the cluster. + In this demo we have chosen to convert them to a Java representation so we can execute server side Java code. This is done different ways in each of the clients, but at a high level we set an attribute `@class` in the client data structure and this maps to the server-side `type-aliases.properties` which then converts the JSON objects to their relevant Java class. + + ```bash + Trade=com.oracle.coherence.demo.model.Trade + Price=com.oracle.coherence.demo.model.Price + ``` + + If you are using the one client to access data, you do not have to have a Java representation. + + A few of the main areas of code have been included below: + + **Define the domain classes** + + The `@serialization.proxy("Price")` defines the Java class that this object will serialize to. + + ```python + @dataclass + @serialization.proxy("Price") + class Price: + symbol: str + price: float + + @dataclass + @serialization.proxy("Trade") + class Trade: + id: str + symbol: str + quantity: int + price: float + + session: Session + prices: NamedCache[str, Price] + trades: NamedCache[str, Trade] + ``` + + **Connect to the Coherence cluster and setup caches** + + ```python + async def init_coherence() -> None: + """ + Initialize Coherence. + + :return: None + """ + global session + global prices + global trades + + # Uses default of localhost:1408 + session = await Session.create() + prices = await session.get_cache("Price") + trades = await session.get_cache("Trade") + ``` + **Display the cache size** + + ```python + async def display_cache_size() -> None: + """ + Displays the size for both the Trade and Price caches. + + :return: None + """ + global prices + global trades + + tradesize = await trades.size() + pricesize = await prices.size() + + print(f"Trade cache size: {tradesize}") + print(f"Price cache size: {pricesize}") + ``` + + **Monitor prices** + + ```python + async def monitor_prices() -> None: + """ + Monitors the Price cache for any changes and displays them. + + :return: None + """ + global prices + + listener: MapListener[str, Price] = MapListener() + listener.on_updated(lambda e: handle_event(e)) + await prices.add_map_listener(listener) + + print("Listening for price changes. Press CTRL-C to finish.") + await asyncio.sleep(10000) + + def handle_event(e) -> None: + """ + Event handler to display the event details + + :return: None + """ + symbol = e.key + old_price = e.old.price + new_price = e.new.price + change = new_price - old_price + + print( + f"Price changed for {symbol}, new=${new_price:.2f}, old=${old_price:.2f}, change=${change:.2f}") + ``` + + **Add trades** + + ```python + async def add_trades(symbol: str, count: int) -> None: + """ + Add trades for a symbol. + + :param symbol the symbol to add trades to + :param count the number of trades to add + :return: None + """ + global prices + global trades + + if count <= 0: + print("count must be supplied and be positive") + return + + # Return a list of the valid symbols from Coherence + symbols: List[str] = await prices.aggregate(Aggregators.distinct("symbol")) + + if symbol in symbols: + current_price: Price = await prices.get(symbol) + + buffer: dict[str, Trade] = {} + print() + + print(f"{get_time()}: Adding {count} random trades for {symbol}") + + for i in range(0, count): + trade_id = str(uuid.uuid1()) + new_trade: Trade = Trade(trade_id, symbol, random.randint(1, 1000), current_price.price) + buffer[trade_id] = new_trade + if i % 1000 == 0: + await trades.put_all(buffer) + buffer.clear() + + # Write anything left + if len(buffer) != 0: + await trades.put_all(buffer) + + size = await trades.size() + print(f"{get_time()}: Size of Trade cache is now {size}") + else: + print(f"Unable to find {symbol}, valid symbols are {symbols}") + ``` + + **Stock split** + + ```python + async def stock_split(symbol: str, factor: int) -> None: + """ + Do a stock split. + + :param symbol the symbol to split + :param factor the factor to use for the split, e.g. 2 = 2 to 1 + :return: None + """ + global prices + global trades + + if factor <= 0 or factor > 10: + print("factor must be supplied and be positive and less than 10") + return + + symbols: List[str] = await prices.aggregate(Aggregators.distinct("symbol")) + + if symbol in symbols: + current_price: Price = await prices.get(symbol) + + # the process for the stock split is: + # 1. Update each trade and multiply the quantity by thr factor + # 2. Update each trade and divide the price by the factor (or multiply by 1/factor) + # 3. Update the price cache for the symbol and divide the price by the factor (or multiply by 1/factor) + + print() + print(f"{get_time()}: Splitting {symbol} using factor of {factor}") + + print(f"{get_time()}: Update quantity for {symbol}") + async for _ in trades.invoke_all(Processors.multiply("quantity", factor), None, Filters.equals("symbol", symbol)): + break # ignore + + print(f"{get_time()}: Update price for {symbol}") + async for _ in trades.invoke_all(Processors.multiply("price", 1 / factor), None, Filters.equals("symbol", symbol)): + break # ignore + + await prices.invoke(symbol, Processors.multiply("price", 1 / factor)) + + new_price = (current_price.price / factor) + + print(f"{get_time()}: Updating price for {symbol} to ${new_price:.2f}") + else: + print(f"Unable to find {symbol}, valid symbols are {symbols}") + ``` + +## Task 5: Explore the Javascript Code + +The **JavaScript** code is available in the following location: + + **clients/js/main.js**
+ **[main.js](https://github.com/coherence-community/coherence-demo/blob/1412/clients/js/main.js)** + +1. Define the domain classes + ```javascript + // create a Trade + function createTrade(symbol, qty, price) { + const trade = { + '@class': 'Trade', + id: uuid.v4().toString(), + symbol: symbol, + quantity: qty, + price: price + } + + return trade + } + ``` + + No actual classes / objects are required in JavaScript, as you can work directly with JSON. + +2. Connect to the Coherence cluster and setup caches + + ```javascript + // setup session to Coherence + const session = new Session() + const prices = session.getCache('Price') + const trades = session.getCache('Trade') + ``` + +3. Display the cache size + + ```javascript + if (command === "size") { + console.log("Trade cache size = " + (await trades.size)) + console.log("Price cache size = " + (await prices.size)) + } + ``` + +4. Monitor prices + + ```javascript + // monitor any price changes + async function monitor() { + console.log("Listening for price changes. Press CTRL-C to finish.") + const handler = (event) => { + let oldPrice = event.oldValue.price; + let newPrice = event.newValue.price; + let change = newPrice - oldPrice; + + console.log("Price changed for " + event.key + ", new=" + formatter.format(newPrice) + + ", old=" + formatter.format(oldPrice) + ", change=" + formatter.format(change)) + } + const listener = new MapListener().on(MapEventType.UPDATE, handler) + + await prices.addMapListener(listener) + await sleep(100_000_000) + } + ``` + +5. Add trades + + ```javascript + // add a number of trades for a symbol + async function addTrades(symbol, count) { + if (count < 0) { + console.log("Count must not be negative") + return + } + + // get the distinct list of symbols + let symbols = await prices.aggregate(Aggregators.distinct('symbol')) + + if (!symbols.includes(symbol, 0)) { + console.log("Unable to find " + symbol + ", valid values are " + symbols) + return + } + + console.log(new Date().toISOString() + ": Adding %d random trades for %s...", count, symbol) + + // get the current price for the trade + let currentPrice = await prices.get(symbol) + + // use efficient setAll (equivalent of putAll) + let buffer = new Map() + for (let i = 0; i < count; i++) { + let trade = createTrade(symbol, Math.floor(Math.random() * 1000), currentPrice.price) + buffer.set(trade.id, trade) + if (i % 1000 === 0) { + await trades.setAll(buffer) + buffer.clear() + } + } + + if (buffer.size !== 0) { + await trades.setAll(buffer) + } + + let size = await trades.size + console.log(new Date().toISOString() + ": Trades cache size is now " + size) + } + ``` + +6. Stock split + + ```javascript + // split a stock using a given factor + async function stockSplit(symbol, factor) { + if (factor < 0) { + console.log("Factor must not be negative") + return + } + + let symbols = await prices.aggregate(Aggregators.distinct('symbol')) + + if (!symbols.includes(symbol, 0)) { + console.log("Unable to find " + symbol + ", valid values are " + symbols) + return + } + + console.log(new Date().toISOString() + ": Splitting %s using factor of %d...", symbol, factor) + + // get the current price for the trade + let currentPrice = await prices.get(symbol) + + // the process for the stock split is: + // 1. Update each trade and multiply the quantity by thr factor + // 2. Update each trade and divide the price by the factor (or multiply by 1/factor) + // 3. Update the price cache for the symbol and divide the price by the factor (or multiply by 1/factor) + + let filter = Filters.equal("symbol", symbol) + + console.log(new Date().toISOString() + ": Updating quantity for " + symbol + " trades...") + await trades.invokeAll(filter, Processors.multiply("quantity", factor)) + + console.log(new Date().toISOString() + ": Updating price for " + symbol + " trades...") + await trades.invokeAll(filter, Processors.multiply("price", 1 / factor)) + + let newPrice = (currentPrice.price / factor) + + console.log(new Date().toISOString() + ": Updating price for " + symbol + " to " + formatter.format(newPrice)) + await prices.invoke(symbol, Processors.multiply("price", 1 / factor)) + } + ``` + +## Task 6: Explore the Go Code + +The **Go code** is available in the following location: + + **clients/go/main.go**
+ **[main.go](https://github.com/coherence-community/coherence-demo/blob/1412/clients/go/main.go)** + +1. Define the domain classes + + ```go + type Trade struct { + Class string `json:"@class"` + ID string `json:"id"` + Symbol string `json:"symbol"` + Quantity int `json:"quantity"` + Price float32 `json:"price"` + } + + type Price struct { + Class string `json:"@class"` + Symbol string `json:"symbol"` + Price float32 `json:"price"` + } + ``` + +2. Connect to the Coherence cluster and setup caches + + ```go + // create a new Session + session, err := coherence.NewSession(ctx, coherence.WithPlainText(), coherence.WithRequestTimeout(time.Duration(120)*time.Second)) + err != nil { + panic(err) + } + defer session.Close() + + trades, err = coherence.GetNamedCache[string, Trade](session, "Trade") + if err != nil { + panic(err) + } + + prices, err = coherence.GetNamedCache[string, Price](session, "Price") + if err != nil { + panic(err) + } + ``` + +3. Display the cache size + + ```go + func displaySize(trades coherence.NamedCache[string, Trade], prices coherence.NamedCache[string, Price]) error { + size, err := trades.Size(ctx) + if err != nil { + return err + } + fmt.Printf("Trade cache size = %d\n", size) + + size, err = prices.Size(ctx) + if err != nil { + return err + } + fmt.Printf("Price cache size = %d\n\n", size) + return nil + } + ``` + +4. Monitor prices + + ```go + func listenPrices(prices coherence.NamedCache[string, Price]) error { + fmt.Println("Listening for price changes. Press CTRL-C to finish.") + fmt.Println() + + // Create a listener and add to the cache + listener := coherence.NewMapListener[string, Price]().OnUpdated(func(e coherence.MapEvent[string, Price]) { + key, err := e.Key() + if err != nil { + panic(err) + } + newValue, err := e.NewValue() + if err != nil { + panic(err) + } + oldValue, err := e.OldValue() + if err != nil { + panic(err) + } + + newPrice := newValue.Price + oldPrice := oldValue.Price + change := newPrice - oldPrice + log.Printf("Price changed for %s, new=$%3.2f, old=$%3.2f, change=$%3.2f\n", *key, oldPrice, newPrice, change) + }) + + if err := prices.AddListener(ctx, listener); err != nil { + return err + } + + select {} + ``` + +5. Add trades + + ```go + func addTrades(trades coherence.NamedCache[string, Trade], prices coherence.NamedCache[string, Price], options ...string) error { + if len(options) != 2 { + return fmt.Errorf("you must specify a symbol and count") + } + + symbol := options[0] + count, err := strconv.Atoi(options[1]) + if err != nil { + return fmt.Errorf("invalid value for count of %v", options[1]) + } + + if count < 0 { + return errors.New("count cannot be negative") + } + + symbols, err := getSymbols(prices) + if err != nil { + return err + } + + if !isSymbolValid(symbol, symbols) { + return fmt.Errorf("unable to find symbol %s, valid values are %v\n", symbol, symbols) + } + + // get the price for the symbol + currentPrice, err := prices.Get(ctx, symbol) + if err != nil { + return err + } + + // add using efficient PutAll + buffer := make(map[string]Trade, 0) + + log.Printf("Adding %d random trades for %s...\n", count, symbol) + + for i := 0; i < count; i++ { + trade := newTrade(symbol, rand.Intn(1000)+1, currentPrice.Price) + buffer[trade.ID] = trade + if i%1000 == 0 { + err = trades.PutAll(ctx, buffer) + if err != nil { + return err + } + buffer = make(map[string]Trade, 0) + } + } + + // if anything left in buffer save to Coherence + if len(buffer) > 0 { + err = trades.PutAll(ctx, buffer) + if err != nil { + return err + } + } + + size, err := trades.Size(ctx) + if err == nil { + log.Printf("Trades cache size is now %d\n", size) + fmt.Println() + } + + return nil + } + ``` + +6. Stock split + + ```go + func stockSplit(trades coherence.NamedCache[string, Trade], prices coherence.NamedCache[string, Price], options ...string) error { + if len(options) != 2 { + return fmt.Errorf("you must specify a symbol and factor") + } + + symbol := options[0] + factor, err := strconv.Atoi(options[1]) + if err != nil { + return fmt.Errorf("invalid value for factor of %v", options[1]) + } + + if factor < 1 || factor > 10 { + return errors.New("factor must be between 1 and 10") + } + + symbols, err := getSymbols(prices) + if err != nil { + return err + } + + if !isSymbolValid(symbol, symbols) { + return fmt.Errorf("unable to find symbol %s, valid values are %v\n", symbol, symbols) + } + + // get the price for the symbol + currentPrice, err := prices.Get(ctx, symbol) + if err != nil { + return err + } + + // the process for the stock split is: + // 1. Update each trade and multiply the quantity by the factor + // 2. Update each trade and divide the price by the factor (or multiply by 1/factor) + // 3. Update the price cache for the symbol and divide the price by the factor (or multiply by 1/factor) + + symbolExtractor := extractors.Extract[string]("symbol") + + ch := coherence.InvokeAllFilter[string, Trade, string](ctx, trades, filters.Equal(symbolExtractor, symbol), + processors.Multiply("quantity", factor)) + + count := 0 + for v := range ch { + if v.Err != nil { + return v.Err + } + count++ + } + + log.Printf("Updated quantity for %d trades", count) + + count = 0 + ch2 := coherence.InvokeAllFilter[string, Trade, string](ctx, trades, filters.Equal(symbolExtractor, symbol), + processors.Multiply("price", float32(1)/float32(factor))) + + for v := range ch2 { + if v.Err != nil { + return v.Err + } + count++ + } + + log.Printf("Updated price for %d trades", count) + + _, err = coherence.Invoke[string, Price, float32](ctx, prices, symbol, processors.Multiply("price", float32(1)/float32(factor))) + + log.Printf("Updated price for %s from $%3.2f to $%3.2f\n\n", symbol, currentPrice.Price, currentPrice.Price/float32(factor)) + // update the price cache + return nil + } + ``` + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/explore-code/images/directory.png b/coherence-demo/explore-code/images/directory.png new file mode 100644 index 0000000..4b48dc2 Binary files /dev/null and b/coherence-demo/explore-code/images/directory.png differ diff --git a/coherence-demo/explore-code/images/files.png b/coherence-demo/explore-code/images/files.png new file mode 100644 index 0000000..5f9e4d5 Binary files /dev/null and b/coherence-demo/explore-code/images/files.png differ diff --git a/coherence-demo/explore/explore.md b/coherence-demo/explore/explore.md new file mode 100644 index 0000000..84af3b2 --- /dev/null +++ b/coherence-demo/explore/explore.md @@ -0,0 +1,173 @@ +# Explore the Demonstration Application + +## Introduction + +This lab walks you through starting the fictitious stock application and exploring application functionality. + +The application is web-based, using Angular and Bootstrap, with the UI accessing Coherence via JAX-RS endpoints. +For this Lab it is run standalone on a VM but can also be deployed to Kubernetes via the [Coherence Operator](https://github.com/oracle/coherence-operator). + +The application uses the “Oracle Bedrock” framework to start/stop additional Coherence processes as well as a secondary cluster for Federation. + +By default, the application uses Coherence Community Edition (CE), but for the purposes of this demonstration we are running it using Grid Edition (14.1.2.0.0) +to showcase Federation capabilities. + +![Coherence Demo](images/screenshot.png "Coherence Demo Application") + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Start the application +* Explore application “Insight” +* Add and split trades +* Scale and shrink the Coherence cluster +* Listen for events + +### Prerequisites + +This lab assumes you have: + +* An Oracle Free Tier(Trial), Paid or LiveLabs Cloud Account +* You have completed: + * Lab: Prepare Setup (Free-tier and Paid Tenants only) + +The following has already been setup in this VM: + +1. The application has been cloned from the GitHub repository `https://github.com/coherence-community/coherence-demo`. +2. JDK21 and Maven version 3.8.8 has already been installed +3. The application has already been built using Maven + +## Task 1: Start the Application + +1. Open a new terminal and change to the `coherence-demo` directory and verify the environment. + + ```bash + cd ~/coherence-demo + mvn -v + ``` + + You will have output similar to the following: + + ```bash + Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39) + Maven home: /home/opc/Downloads/apache-maven-3.8.8 + Java version: 21.0.5, vendor: Oracle Corporation, runtime: /usr/lib/jvm/jdk-21.0.5-oracle-x64 + Default locale: en_US, platform encoding: UTF-8 + OS name: "linux", version: "5.15.0-104.119.4.2.el8uek.x86_64", arch: "amd64", family: "unix" + ``` + +2. In the same terminal, issue the following command to start the application: + + ```bash + mvn exec:exec -P grid-edition + ``` + + You should see a screen similar to the following showing the welcome message. Click `Close` to continue. + + ![Splash Screen](images/splash-screen.png "Splash Screen") + +3. View the application home page. In the browser window, you will see four application panels.
+ + **Portfolio Composition** shows the different stock symbols with fictitious prices.
+ **Aggregation Performance** shows the time taken to retrieve the aggregated data for the first panel.
+ **Data Distribution** shows the data distribution of the stock data amongst members.
+ **Cluster Management** shows the current members and allows you to start and stop members.
+ +4. There are also three menus, **`Federation`**, **`Persistence`** and **`Tools`** that we explore later. + +5. Use the **`Tools`** menu and select **`About`**. This will show a screen similar to the following with information about the application including the Coherence version. + ![About](images/about.png "About") + + + +## Task 2: Explore application “Insight” + +1. **"Insight"** Icons + Throughout the application, there are small **i** icons which allows you to view "Insight" into different application aspects. + For example, the following insight icon on the "Portfolio Composition" panel, shows the code that is run to aggregate the portfolio. + + ![Demo Insight](images/demo-insight.png "Insight") + +2. After each operation, such as adding trades or starting servers for example, "Insight" is also displayed to explain the operation or show relevant code. + +3. If you wish to disable "Insight" entirely, choose **`Tools`** -> **`Disable Insight`**. You can re-enable it in the same menu. + +## Task 3: Add and split trades + +In this task we will add and split trades. + +1. Click the **`gear`** icon next to the **`ORCL`** stock and click **`Add Trades`**. + + ![Add ORCL Trades](images/add-orcl-trades.png "Add ORCL Trades") + + ![Add Trades](images/add-trades.png "Add Trades") + + You should see the number of trades increase to 101,000. + +2. In the same way, do a stock split for **`ORCL`** enter **`2`** as the factor. When this is complete, you will see a popup dialog showing the code that was run to split the trades. + +## Task 4: Scale and shrink the Coherence cluster + +In this step we will scale the Coherence by an addition 3 members to a total of four members. When new storage members are added the cache data is automatically and transparently balanced amongst the available servers for high-availability and scalability. + +As this scaling is taking place, we can see that the data stays consistent and available when it is being balanced. + +Each cache entry has a primary and at least one backup which is stored as far away from the primary as possible and always on a separate cache node. + +Depending upon your topology it may also be on a separate machine, rack or site to provide additional high-availability. + +As more cache servers are added there is additional memory capacity as well as processing capacity allowing parallel queries to more efficiently run. + +1. On the bottom right panel click **`Add Servers`** and enter **3** as the number of addition servers to add. + + ![Add Servers](images/add-servers.png "Add Servers") + +2. You will notice the data distribution changing and then finally stabilizing at 4 servers with each server holding an almost equal portion of the data. + + ![More Servers](images/initial-balanced.png "More Servers") + +3. You will also see the time to perform the aggregation will reduce due to the addition cache server resources available to process the aggregations. + > Note: If you scale the servers too high in this VM environment, the times will not continue increasing due to CPU constraints. + +4. Select **`Server 4`** on the bottom right panel and select **`Stop Server`**. This will terminate the server immediately simulating a failure. Coherence will automatically detect this failre and re-balance the data while the application is continuing without interruption keeping the data consistent. Any missing primary data will have their backups promoted to primary and new backups made for missing backups. + + ![Stop Server](images/stop-server.png "Stop Server") + +5. The updated distribution is shown below: + + ![Updated Distribution](images/updated-distribution.png "Updated Distribution") + +## Task 5: Listen for events + +Coherence allows you to listen and response to cache entry inserts, updates and deletes for a specific key, a filter or all entries. + +In the demo we enable random price updates and listen via a **"Map Listener"** and expose this over Server Sent Events (SSE) to the web client. + +Map Listeners, in this example are written in Java code, but can also be applied using JavaScript, Python or Go. + +See [here](https://github.com/coherence-community/coherence-demo/blob/cacf32ca2550032862cdf20bcef2b43c145b7794/src/main/java/com/oracle/coherence/demo/application/EventsResource.java#L66) for the source code for this event registration. + +1. Click on the checkbox next to **`Real-time Price Updates`** to enable random stock price updates. + + ![Price Updates](images/price-updates.png "Price Updates") + + You should now see the prices changing. + +2. From **`Tools Menu`** choose **`Monitor Prices`**, this will open a new windows using **“Server Sent Events”**. You should see a window similar to the following: + + ![Monitor Prices](images/sse.png "Monitor Prices") + +## Learn More + +* [Coherence Demo Source](https://github.com/coherence-community/coherence-demo) +* [Coherence Operator for Kubernetes](https://github.com/oracle/coherence-operator) +* [Oracle Bedrock framework](https://github.com/coherence-community/oracle-bedrock) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/explore/images/about.png b/coherence-demo/explore/images/about.png new file mode 100644 index 0000000..7c56841 Binary files /dev/null and b/coherence-demo/explore/images/about.png differ diff --git a/coherence-demo/explore/images/add-orcl-trades.png b/coherence-demo/explore/images/add-orcl-trades.png new file mode 100644 index 0000000..f571906 Binary files /dev/null and b/coherence-demo/explore/images/add-orcl-trades.png differ diff --git a/coherence-demo/explore/images/add-servers.png b/coherence-demo/explore/images/add-servers.png new file mode 100644 index 0000000..eccc900 Binary files /dev/null and b/coherence-demo/explore/images/add-servers.png differ diff --git a/coherence-demo/explore/images/add-trades.png b/coherence-demo/explore/images/add-trades.png new file mode 100644 index 0000000..c9b0444 Binary files /dev/null and b/coherence-demo/explore/images/add-trades.png differ diff --git a/coherence-demo/explore/images/demo-insight.png b/coherence-demo/explore/images/demo-insight.png new file mode 100644 index 0000000..306ac62 Binary files /dev/null and b/coherence-demo/explore/images/demo-insight.png differ diff --git a/coherence-demo/explore/images/initial-balanced.png b/coherence-demo/explore/images/initial-balanced.png new file mode 100644 index 0000000..c7898d3 Binary files /dev/null and b/coherence-demo/explore/images/initial-balanced.png differ diff --git a/coherence-demo/explore/images/price-updates.png b/coherence-demo/explore/images/price-updates.png new file mode 100644 index 0000000..da5b91f Binary files /dev/null and b/coherence-demo/explore/images/price-updates.png differ diff --git a/coherence-demo/explore/images/screenshot.png b/coherence-demo/explore/images/screenshot.png new file mode 100644 index 0000000..01633f9 Binary files /dev/null and b/coherence-demo/explore/images/screenshot.png differ diff --git a/coherence-demo/explore/images/splash-screen.png b/coherence-demo/explore/images/splash-screen.png new file mode 100644 index 0000000..81bb5fa Binary files /dev/null and b/coherence-demo/explore/images/splash-screen.png differ diff --git a/coherence-demo/explore/images/sse.png b/coherence-demo/explore/images/sse.png new file mode 100644 index 0000000..03b2e4b Binary files /dev/null and b/coherence-demo/explore/images/sse.png differ diff --git a/coherence-demo/explore/images/stop-server.png b/coherence-demo/explore/images/stop-server.png new file mode 100644 index 0000000..13846b5 Binary files /dev/null and b/coherence-demo/explore/images/stop-server.png differ diff --git a/coherence-demo/explore/images/updated-distribution.png b/coherence-demo/explore/images/updated-distribution.png new file mode 100644 index 0000000..01cb089 Binary files /dev/null and b/coherence-demo/explore/images/updated-distribution.png differ diff --git a/coherence-demo/federate/federate.md b/coherence-demo/federate/federate.md new file mode 100644 index 0000000..f4c8e3c --- /dev/null +++ b/coherence-demo/federate/federate.md @@ -0,0 +1,68 @@ +# Federate Data Between Clusters + +## Introduction + +This lab walks you through starting a secondary cluster and using federation to replicate data between the two clusters. + +Federated caching federates cache data asynchronously across multiple geographically dispersed clusters. Cached data is federated across clusters to **provide redundancy**, **off-site backup**, and **multiple points of access** for application users in different geographical locations. + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Enable federation +* Open the secondary cluster +* Add data in both clusters +* Enable price updates + +### Prerequisite + +* You should have completed the previous labs. + +## Task 1: Start Federation + +> *Note: Ensure you have disabled price updates before continuing*. + +1. From the **`Federation`** menu, select **`Start Federation`**. After a short time you should see “Data sent to remote cluster..” and a cluster name appearing at the top as show here. The cluster name is chosen based upon your region location. + ![Federation Start](images/federation-header.png "Federation Start") + > If you have **"Insight"** enabled, a dialog will be displayed showing how federation is setup in configuration only. No actual application code changes are required. + +2. Select **`Dashboard`** on the **`Federation`** menu to open the secondary cluster dashboard. In our case the secondary cluster is "San Francisco". + ![Open Secondary](images/secondary.png "Open Secondary") + + > You should see the same data on the secondary dashboard. Stop the price updates on the first cluster and see it should match. + + ![Open Secondary](images/secondary-cluster.png "Open Secondary") + +## Task 2: Add Trades to the Primary Cluster + +1. Switch back to the primary cluster dashboard and use **`Tools`** -> **`Add Trades`** to add 10,000 trades. You will see these be added to the secondary cluster. + + ![Add Trades](images/add-trades.png "Add Trades") + +2. Switch to the secondary cluster dashboard, and you will see the trades replicated to this cluster. + +## Task 3: Add Trades to the Secondary Cluster + +1. On the secondary cluster dashboard, use **`Tools`** -> **`Add Trades`** to add 5,000 trades. Because we have configured federation to be active-active, +you will see these trades added to the primary cluster as well. + +## Task 4: Enable Price Updates in the Primary Cluster + +1. On the primary cluster, Click on the checkbox next to `Real-time Price Updates` to enable random stock price updates. + + ![Price Updates](images/price-updates.png "Price Updates") + + You should now see the prices changing and also being replicated to the secondary cluster. + +## Learn More + +* [Federation Documentation](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.1.2206/administer/federating-caches-clusters.html) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/federate/images/add-trades.png b/coherence-demo/federate/images/add-trades.png new file mode 100644 index 0000000..c9cc284 Binary files /dev/null and b/coherence-demo/federate/images/add-trades.png differ diff --git a/coherence-demo/federate/images/federation-header.png b/coherence-demo/federate/images/federation-header.png new file mode 100644 index 0000000..024354c Binary files /dev/null and b/coherence-demo/federate/images/federation-header.png differ diff --git a/coherence-demo/federate/images/price-updates.png b/coherence-demo/federate/images/price-updates.png new file mode 100644 index 0000000..da5b91f Binary files /dev/null and b/coherence-demo/federate/images/price-updates.png differ diff --git a/coherence-demo/federate/images/secondary-cluster.png b/coherence-demo/federate/images/secondary-cluster.png new file mode 100644 index 0000000..5e7d3dc Binary files /dev/null and b/coherence-demo/federate/images/secondary-cluster.png differ diff --git a/coherence-demo/federate/images/secondary.png b/coherence-demo/federate/images/secondary.png new file mode 100644 index 0000000..ad750f2 Binary files /dev/null and b/coherence-demo/federate/images/secondary.png differ diff --git a/coherence-demo/intro/intro.md b/coherence-demo/intro/intro.md new file mode 100644 index 0000000..29e8997 --- /dev/null +++ b/coherence-demo/intro/intro.md @@ -0,0 +1,53 @@ +# Introduction + +## About this Workshop + +This workshop walks through the "Coherence Demonstration Application", which is a fictitious stock application, +which demonstrates general Coherence features, scalability capabilities including: + +* Clustering and Data Sharding +* Scalability and High Availability +* Disk-Based Persistence +* Parallel Queries +* Efficient Aggregation +* In-Place Processing +* Federation (Grid Edition feature only) +* OpenTracing Support +* Polyglot client access from JavaScript, Python and Golang +* Listening for events using Server Sent Events (SSE) + +Estimated time: 60 minutes + + +### Objectives + +In this workshop, you will: + +* Run the Coherence Demonstration application showing various Coherence features +* Replicate data between multiple clusters using Federation +* Access the application via JavaScript, Python and Go clients +* Use OpenTracing to gain insight into a Coherence cluster from the API down to the Database +* Persist and recover cache data from disk + +### Prerequisites + +This lab assumes you have: + +* An Oracle Cloud Account - Please view this workshop's LiveLabs landing page to see which environments are supported + +## About Oracle Coherence + +Oracle Coherence is an in-memory data grid enables application developers and managers fast access to key-value data. +Coherence ensures for customers maximum scalability and performance in enterprise applications by providing clustered +low-latency data storage, polyglot grid computing, and asynchronous event streaming. + +## Learn More + +* [Coherence Documentation](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.2.0/index.html) +* [Coherence Community](https://coherence.community/) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/open-tracing/images/console.png b/coherence-demo/open-tracing/images/console.png new file mode 100644 index 0000000..e37fa80 Binary files /dev/null and b/coherence-demo/open-tracing/images/console.png differ diff --git a/coherence-demo/open-tracing/images/enable-tracing.png b/coherence-demo/open-tracing/images/enable-tracing.png new file mode 100644 index 0000000..186329c Binary files /dev/null and b/coherence-demo/open-tracing/images/enable-tracing.png differ diff --git a/coherence-demo/open-tracing/images/expand.png b/coherence-demo/open-tracing/images/expand.png new file mode 100644 index 0000000..f5af6d1 Binary files /dev/null and b/coherence-demo/open-tracing/images/expand.png differ diff --git a/coherence-demo/open-tracing/images/narrow.png b/coherence-demo/open-tracing/images/narrow.png new file mode 100644 index 0000000..89daefa Binary files /dev/null and b/coherence-demo/open-tracing/images/narrow.png differ diff --git a/coherence-demo/open-tracing/images/select-service.png b/coherence-demo/open-tracing/images/select-service.png new file mode 100644 index 0000000..c6f2fbd Binary files /dev/null and b/coherence-demo/open-tracing/images/select-service.png differ diff --git a/coherence-demo/open-tracing/images/select-trace.png b/coherence-demo/open-tracing/images/select-trace.png new file mode 100644 index 0000000..ae4c596 Binary files /dev/null and b/coherence-demo/open-tracing/images/select-trace.png differ diff --git a/coherence-demo/open-tracing/images/step-1.png b/coherence-demo/open-tracing/images/step-1.png new file mode 100644 index 0000000..1629ce0 Binary files /dev/null and b/coherence-demo/open-tracing/images/step-1.png differ diff --git a/coherence-demo/open-tracing/images/step-2.png b/coherence-demo/open-tracing/images/step-2.png new file mode 100644 index 0000000..6e128fd Binary files /dev/null and b/coherence-demo/open-tracing/images/step-2.png differ diff --git a/coherence-demo/open-tracing/images/step-3.png b/coherence-demo/open-tracing/images/step-3.png new file mode 100644 index 0000000..04b48e4 Binary files /dev/null and b/coherence-demo/open-tracing/images/step-3.png differ diff --git a/coherence-demo/open-tracing/images/step-4.png b/coherence-demo/open-tracing/images/step-4.png new file mode 100644 index 0000000..97d99c0 Binary files /dev/null and b/coherence-demo/open-tracing/images/step-4.png differ diff --git a/coherence-demo/open-tracing/images/step-5.png b/coherence-demo/open-tracing/images/step-5.png new file mode 100644 index 0000000..c261917 Binary files /dev/null and b/coherence-demo/open-tracing/images/step-5.png differ diff --git a/coherence-demo/open-tracing/images/tracing-enabled.png b/coherence-demo/open-tracing/images/tracing-enabled.png new file mode 100644 index 0000000..fc0faa6 Binary files /dev/null and b/coherence-demo/open-tracing/images/tracing-enabled.png differ diff --git a/coherence-demo/open-tracing/open-tracing.md b/coherence-demo/open-tracing/open-tracing.md new file mode 100644 index 0000000..9b6b051 --- /dev/null +++ b/coherence-demo/open-tracing/open-tracing.md @@ -0,0 +1,129 @@ +# Gain Insight through OpenTelemetry or OpenTracing APIs + +## Introduction + +This lab will walk you through using OpenTracing to gain visibility in cache operations from the UI, to the cache server and all the way through to the cache store write. + +Coherence does not include any tracing implementation libraries. Therefore, the developer needs to provide the desired tracing runtime. Since OpenTracing is no longer maintained, Oracle recommends that you use OpenTelemetry. A minimum version of OpenTelemetry for Java version 1.29 is recommended. Even though OpenTracing is deprecated in Coherence, it is still a supported option when using the latest OpenTracing 0.33.0. + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Start the Jaegar Docker image +* Enable tracing in the primary cluster +* Trace the update price operation from the JAX-RS endpoint through the cache and to the Database + +### Prerequisites + +* You should have completed the previous labs. + +## Task 1: Start the Jaegar Docker Image + +1. Click on the checkbox next to **`Real-time Price Updates`** to enable random stock price updates. + +2. Open a new terminal by using **`File`** -> **`New Tab`** in the existing terminal. + +3. Start the Jaegar image using the following command: + + ```bash + docker run --rm -d --name jaeger \ + -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \ + -p 5775:5775/udp \ + -p 6831:6831/udp \ + -p 6832:6832/udp \ + -p 5778:5778 \ + -p 16686:16686 \ + -p 14268:14268 \ + -p 9411:9411 \ + jaegertracing/all-in-one:1.20 + ``` + +## Task 2: Configure Tracing on the Primary Cluster + +1. Access the primary cluster console and choose **`Tools`** -> **`Configure Tracing`**, and then choose **`Enable Tracing for Cluster`**. + + ![Enable Tracing](images/enable-tracing.png "Enable Tracing") + +2. You should see in the bottom right panel that tracing has been enabled for each cluster member. + + ![Tracing Enabled](images/tracing-enabled.png "Tracing Enabled") + + > Note: Wait a few minutes to ensure tracing has been picked up by the Docker image. + +## Task 3: Search for Traces + +1. Open the Jaegar console at `http://localhost:16686`. + ```bash + http://localhost:16686 + ``` + + ![Jaegar Console](images/console.png "Jaegar Console") + +2. Select your primary cluster service (Cluster) name and choose operation **`CharDataSource.getChartData`** and click **`Find Traces`**: + + The **`CharDataSource.getChartData`** is the JAX-RS endpoint that will update prices on each refresh when real-time price updates are enabled. + + ![Select Service](images/select-service.png "Select Service") + +3. Select the first trace in the list: + + ![Select Trace](images/select-trace.png "Select Trace") + +4. Narrow down the time by dragging and selecting the light blue panel just below the **"Trace Start"** message. + + ![Narrow Trace](images/narrow.png "Narrow Trace") + +## Task 4: Inspect and Understand the Traces + +> *Note: If you don't see the traces as described below, wait a few minutes and ensure you have enabled price updates.* + +For this task there is a number, e.g.`[1]` in a square bracket on the image below the corresponds to each of the steps below: + + ![Expand](images/expand.png "Expand") + +1. Select **`CharDataSource.getChartData`** and this will show the details of the JAX-RS endpoint request as shown below: + + ![Step 1](images/step-1.png "Step 1") + +2. Click on **`Utilities.UpdatePrices`** and then **`Invoke.request`** and expand the **`Tags`** on the **`Invoke.request`**. This shows
+ The invocation of the entry processor to do the price updates. Entry processors allow you to send processing to where + the data is rather than retrieving the data. This is much more efficient as you are not sending and receiving the data, just the logic to process it. + + ![Step 2](images/step-2.png "Step 2") + + From the above you can see the invoke request was processed on member 1. This may be different in your environment. + +3. Click on the **`Invoke.process`** and expand the **`Tags`**. This shows where the invocation actually executed and is where the primary data resides. This may be any of the members but in our case is also member 1: + + ![Step 3](images/step-3.png "Step 3") + +4. Expand the **`BackupAll.request`** and the **`BackupAll.process`**. + + > The **`BackupAll.request`** is the member on which the primary data resides and this member requests the backup. In our case, member 1. + The **`BackupAll.process`** is where the backup is processed and located, which in our case is member 2.
+ This shows how Coherence will not locate the primary and backup data in the same place to ensure high-availability. + + ![Step 4](images/step-4.png "Step 4") + +5. Expand the **`cachestore.store`** and **`Udpate`** which shows the cache entries being written to a database.
+ For this cache we have a **`Cache Store`** defined, which allows us to keep the database and cache in sync. In this case we have + an in memory DB for convenience, but this could be any JDBC compliant database. + + ![Step 5](images/step-5.png "Step 5") + + > Note: We can see the SQL statement being executed on member 1 where the primary data resides. + +## Learn More + +* [Peek into Coherence using OpenTracing - Blog](https://blogs.oracle.com/oraclecoherence/post/peek-inside-coherence-with-opentracing) +* [Coherence and OpenTracing - Documentation](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.2.0/develop-applications/debugging-coherence.html) +* [Caching Data Sources](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.2.0/develop-applications/caching-data-sources.htm) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, December 2024 \ No newline at end of file diff --git a/coherence-demo/persistence/images/clear-cache.png b/coherence-demo/persistence/images/clear-cache.png new file mode 100644 index 0000000..0275c1a Binary files /dev/null and b/coherence-demo/persistence/images/clear-cache.png differ diff --git a/coherence-demo/persistence/images/create-snapshot.png b/coherence-demo/persistence/images/create-snapshot.png new file mode 100644 index 0000000..77a4097 Binary files /dev/null and b/coherence-demo/persistence/images/create-snapshot.png differ diff --git a/coherence-demo/persistence/persistence.md b/coherence-demo/persistence/persistence.md new file mode 100644 index 0000000..145006a --- /dev/null +++ b/coherence-demo/persistence/persistence.md @@ -0,0 +1,61 @@ +# Work with Persistence + +## Introduction + +This lab will walk you through using Persistence with Coherence. + +Coherence persistence is a set of tools and technologies that manage the persistence and recovery of +Coherence distributed caches. Cached data is persisted so that it can be quickly recovered after a +catastrophic failure or after a cluster restart due to planned maintenance. Persistence and federated caching can be used together as required. + +Estimated time: 10 minutes + +### Objectives + +In this lab, you will: + +* Create a snapshot of the cache data +* Clear the cache data +* Recover the snapshot +* Replicate data to the secondary cluster + +### Prerequisites + +* You should have completed the previous labs. + +## Task 1: Create a Snapshot + +1. Switch to the primary cluster dashboard and use **`Persistence`** -> **`Create Snapshot`** to create a snapshot of the current cache data. + Wait for confirmation that the snapshot has been created before continuing. Note the count of the `Positions`. + + ![Create Snapshot](images/create-snapshot.png "Create Snapshot") + + > Note: Coherence will ensure that the data in the snapshot is consistent. + +## Task 2: Clear the primary data + +1. Use **`Tools`** -> **`Clear Cache`** to remove the cache data from the primary cluster. As we have federation configured, this will be replicated to the secondary cluster. + + ![Clear Cache](images/clear-cache.png "Clear Cache") + + > Note: Confirm that the data has also been removed from the secondary cluster. + +## Task 3: Recover the Snapshot + +1. Switch to the primary cluster dashboard and use **`Persistence`** -> **`Recover Snapshot`** to recover the snapshot data. + + Once the data has been recovered, you will see the updated data in the dashboard. + +2. To ensure that the secondary cluster is updated after recovering a snapshot, you need to re-enable federation by selecting **`Federation`** -> **'Replicate All`**. + + Confirm that the secondary cluster data has also been updated. + +## Learn More + +* [Coherence Persistence Documentation](https://docs.oracle.com/en/middleware/standalone/coherence/14.1.2.0/administer/persisting-caches.html) + +## Acknowledgements + +* **Author** - Tim Middleton +* **Contributors** - Ankit Pandey, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, November 2024 \ No newline at end of file diff --git a/coherence-demo/prepare-setup/images/browse-zip.png b/coherence-demo/prepare-setup/images/browse-zip.png new file mode 100644 index 0000000..cc7688a Binary files /dev/null and b/coherence-demo/prepare-setup/images/browse-zip.png differ diff --git a/coherence-demo/prepare-setup/images/click-stack.png b/coherence-demo/prepare-setup/images/click-stack.png new file mode 100644 index 0000000..a05bbd9 Binary files /dev/null and b/coherence-demo/prepare-setup/images/click-stack.png differ diff --git a/coherence-demo/prepare-setup/images/desktop-url.png b/coherence-demo/prepare-setup/images/desktop-url.png new file mode 100644 index 0000000..57016ad Binary files /dev/null and b/coherence-demo/prepare-setup/images/desktop-url.png differ diff --git a/coherence-demo/prepare-setup/images/instance-shape.png b/coherence-demo/prepare-setup/images/instance-shape.png new file mode 100644 index 0000000..caecb79 Binary files /dev/null and b/coherence-demo/prepare-setup/images/instance-shape.png differ diff --git a/coherence-demo/prepare-setup/images/main-config.png b/coherence-demo/prepare-setup/images/main-config.png new file mode 100644 index 0000000..aa7682a Binary files /dev/null and b/coherence-demo/prepare-setup/images/main-config.png differ diff --git a/coherence-demo/prepare-setup/images/menu-stack.png b/coherence-demo/prepare-setup/images/menu-stack.png new file mode 100644 index 0000000..c640f83 Binary files /dev/null and b/coherence-demo/prepare-setup/images/menu-stack.png differ diff --git a/coherence-demo/prepare-setup/images/run-apply.png b/coherence-demo/prepare-setup/images/run-apply.png new file mode 100644 index 0000000..5cdb79f Binary files /dev/null and b/coherence-demo/prepare-setup/images/run-apply.png differ diff --git a/coherence-demo/prepare-setup/images/select-compartment.png b/coherence-demo/prepare-setup/images/select-compartment.png new file mode 100644 index 0000000..24ba753 Binary files /dev/null and b/coherence-demo/prepare-setup/images/select-compartment.png differ diff --git a/coherence-demo/prepare-setup/prepare-setup.md b/coherence-demo/prepare-setup/prepare-setup.md new file mode 100644 index 0000000..95d13aa --- /dev/null +++ b/coherence-demo/prepare-setup/prepare-setup.md @@ -0,0 +1,82 @@ +# Prepare Setup + +## Introduction +This lab will show you how to download the Oracle Resource Manager (ORM) stack zip file needed to setup the resource needed to run this workshop. Then you creates a compute instance and a Virtual Cloud Network (VCN) which provides you access to a remote desktop. + +Estimated Time: 10 minutes + +### Objectives +* Download ORM stack +* Create Compute + Networking using Resource Manager Stack + +### Prerequisites +This lab assumes you have: +- An Oracle Free Tier or Paid Cloud account + +## Task 1: Download Oracle Resource Manager (ORM) stack zip file + +1. Click on the link below to download the Resource Manager zip file you need to build your environment: + + *Note 1:* If providing a single Stack download for the workshop, use this simple expression. + + - [coherence-orm-mkplc-freetier.zip](https://objectstorage.uk-london-1.oraclecloud.com/p/3kNOQ7B-nbiaEiOMy6_9FekAR4AJXXwQhr6id2s5_JpJAtKm1yT6EhPj2KOZI-ZL/n/lrv4zdykjqrj/b/ankit-bucket/o/coherence-orm-mkplc-freetier.zip) + +2. Save in your downloads folder. + +## Task 2: Create Stack: Compute + Networking + +1. Identify the ORM stack zip file downloaded in **Task 1: Download Oracle Resource Manager (ORM) stack zip file**. + +2. Open up the hamburger menu in the top left corner. Click **Developer Services**, and choose **Resource Manager** > **Stacks**. Choose the compartment in which you would like to install the stack. Click **Create Stack**. + ![menu stack](images/menu-stack.png) + ![select compartment](images/select-compartment.png) + + +3. Select **My Configuration**, choose the **.Zip** file button, click the **Browse** link, and select the zip file that you downloaded or drag-n-drop for the file explorer. Click **Next**. + ![browse zip](images/browse-zip.png) + +4. Enter or select the following and click **Next**. + + **Instance Count:** Accept the default, 1. + + **Select Availability Domain:** Select an availability domain from the dropdown list. + + **Need Remote Access via SSH?** Keep Unchecked for Remote Desktop only Access - The Default. + + **Use Flexible Instance Shape with Adjustable OCPU Count?:** Keep the default as checked (unless you plan on using a fixed shape). + + **Instance Shape:** Keep the default or select from the list of Flex shapes in the dropdown menu (e.g VM.Standard.E4.Flex). + + **Select OCPUs Count per Instance:** Accept the default shown. e.g. (2) will provision 2 OCPUs and 32GB of memory. + + **Use Existing VCN?:** Accept the default by leaving this unchecked. This will create a new VCN. + ![main config](images/main-config.png) + ![instance shape](images/instance-shape.png) + + +7. Select **Run Apply** and click **Create**. + ![run apply](images/run-apply.png) + + + +## Task 3: Access the Graphical Remote Desktop + +For ease of execution of this workshop, your VM instance has been pre-configured with a remote graphical desktop accessible using any modern browser on your laptop or workstation. Proceed as detailed below to log in. + +1. Open up the hamburger menu in the top left corner. Click **Developer Services**, and choose **Resource Manager** > **Stacks**. + +2. Click on the stack name which you have createed in Task 2. + ![click stack](images/click-stack.png) + +3. Navigate to **Application Information** tab, and copy **Remote Desktop URL** and paste it in new browser tab. + ![desktop url](images/desktop-url.png) + + > Now you need to follow all the instruction inside this remote desktop. + + +You may now proceed to the next lab. + +## Acknowledgements +* **Author** - Ankit Pandey +* **Contributors** - Maciej Gruszka, Sid Joshi +* **Last Updated By/Date** - Ankit Pandey, January 2025 diff --git a/coherence-demo/workshops/freetier/index.html b/coherence-demo/workshops/freetier/index.html new file mode 100644 index 0000000..aaac634 --- /dev/null +++ b/coherence-demo/workshops/freetier/index.html @@ -0,0 +1,62 @@ + + + + + + + + + Oracle LiveLabs + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/coherence-demo/workshops/freetier/manifest.json b/coherence-demo/workshops/freetier/manifest.json new file mode 100644 index 0000000..0a5c4bb --- /dev/null +++ b/coherence-demo/workshops/freetier/manifest.json @@ -0,0 +1,56 @@ +{ + "workshoptitle": "Coherence Demonstration Application", + "help": "livelabs-help-weblogic_us@oracle.com", + "tutorials": [ + { + "title": "Introduction", + "type": "freetier", + "filename": "../../intro/intro.md" + }, + { + "title": "Get Started", + "description": "Prerequisites", + "filename": "https://oracle-livelabs.github.io/common/labs/cloud-login/pre-register-free-tier-account.md" + }, + { + "title": "Lab 1: Prepare Setup", + "type": "freetier", + "filename": "../../prepare-setup/prepare-setup.md" + }, + { + "title": "Lab 2: Explore the Demonstration Application", + "type": "freetier", + "filename": "../../explore/explore.md" + }, + { + "title": "Lab 3: Federate Data Between Clusters", + "type": "freetier", + "filename": "../../federate/federate.md" + }, + { + "title": "Lab 4: Explore Python, JavaScript and Go Clients", + "type": "freetier", + "filename": "../../explore-clients/explore-clients.md" + }, + { + "title": "Lab 5: Gain Insight through Open Tracing", + "type": "freetier", + "filename": "../../open-tracing/open-tracing.md" + }, + { + "title": "Lab 6: Work with Persistence", + "type": "freetier", + "filename": "../../persistence/persistence.md" + }, + { + "title": "Lab 7: Explore the Application Code", + "type": "freetier", + "filename": "../../explore-code/explore-code.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename":"https://oracle-livelabs.github.io/common/labs/need-help/need-help-freetier.md" + } + ] +}