Why and how to engage a Complex Event Processor from a Java Web Application
3 likes452 views
This document discusses the integration of complex event processing (CEP) within Java web applications to analyze and manage real-time event streams from various sources. It explains how CEP can detect patterns, exceptions, and calculate aggregates by using continuous query language (CQL) on incoming event data. The document also provides examples of practical applications, such as monitoring temperature sensors and analyzing user interactions in a webshop, emphasizing the benefits of CEP for real-time insights and business intelligence.
![Derive intelligence from WebShop eventsActual Metrics: #searches in last 5 minutes, #buyers in the last 24 hours, #search-to-views drilldownsTop 10 search terms (in last [24] hour)Top 10 useless search termsAfter search on term there is no view eventMost Effective search termsSearch is followed by (one or more) view events and subsequently a buy eventItems frequently viewed but not often bought#buy/#views is ranks low](https://image.slidesharecdn.com/complexeventprocessinginjavaapps-100921182219-phpapp01/85/Why-and-how-to-engage-a-Complex-Event-Processor-from-a-Java-Web-Application-36-320.jpg)
Why and how to engage a Complex Event Processor from a Java Web Application
- 1. Why and How to Engage a Complex Event Processor from a Java Web ApplicationLucas JellemaCTO, AMIS (The Netherlands)Oracle ACE Director
- 2. An EventA factWhen it becomes a factTimestamp!Of potential interest to anyoneWith some information associated with itWhen the fact is changed, that too may constitute an eventNote that even when no change occurs, that could be considered an event (absence of expected event)
- 3. The Data AvalancheThe league of real time eventsContinuous stream of a multitude of tiny events with hardly any payload, most of which are not interesting at allSent from physical sensors (temperature, pressure, RFID, security gates), process sensors, Twitter, manufacturing equipment, database triggers, web servers, ESBs, stock trade tickers, sport statistics, RSS, network switches, …
- 4. Most of these events….
- 5. Making Sense of Non-senseComplex Event ProcessingUseful information could be hidden in those streams of events – and CEP extracts itBy detecting patterns, deviations, aggregations in these event streamsCEP publishes ‘meaningful’ (business) events to report its findings – by executing some Event Processing LangugaeeventsComplex Event Processor
- 6. Continuous Query Language (CQL)Standard extension/complement to SQLFor handling large volumes of continuously arriving eventsExecuting the queries continuouslyOperates on streams of continually arriving events instead of static relational data setsEvents can be enriched with contextResults keep being produced as event streamIs used for comparing events of same type, to findTrends and PatternsExceptionsAggregatesComplex Event ProcessorCQL
- 7. Example CQL QueryReport the average temperatureAggregated over last 60 secondsReported every 30 secondsDetermined per ClusterTemperaturereadingsComplex Event ProcessorAverage temperature values (over 60s, per cluster, every 30s)
- 8. Event Processing NetworkAdapters – consuming or producing eventsChannels – carrying specific event typesProcessors – running CQL
- 9. Complex Event ProcessingAlertJMSDashboardSOA ApplicationDBData WarehouseWebServiceFileeventsComplex Event ProcessorJMSHTTPJMXFileDBEvent Consumer
- 10. eventseventseventsJava Applications interacting with a complex event processor Complex Event ProcessorJava applicationJMS132
- 11. Various CEP offerings availableProgress Software (Apama)Aleri (Aleri and Coral8)IBM (WebSphere Business Events, System S/InfoSphere Streams, ILog rule engine)StreamBase Systems, TIBCO Software (Tibco Business Events) Oracle (CEP - fka BEA WLEvs - and Glassfish OpenESB IEP) UC4 (Senactive) EsperTechJBoss Drools Fusion
- 12. Oracle CEP 11g Architecturefka BEA WLEVS(erver)Design TimeRun TimeCEP Visualizer(web console)Other WebAppEclipsehttpCEPpluginWeb (Bayeux)ServerCEP ServerDBJMS
- 13. Demo: HelloWorld
- 14. The EPN source side of things
- 15. Java Application as Event ConsumereventsComplex Event ProcessorJava applicationJMS
- 17. Case of TemperatureHospitalTemperature sensorsCooling unitsMaternity Ward and Intensive Care unitsFire detectionFrequent readings (subsecond)Monitoring for: trends of threshold crossing, fire!, fault detection
- 18. Demo: Temperature SensorsApplication should display a dashboard for temperature sensor readingsThe user is interested in cluster aggregates , deviations from pre-set values and broken sensorsHow would a Java Application process thousands of events per secondextract and report aggregatesdetect and report missing valuesNote: user wants real time updates – but not more frequently than once per minute
- 19. Technical LayoutTemperature Sensors are all over the placeClustered per room or locationTemperature Readings arrive over JMSResults are sent through JMS to dashboardJMS queue for temperature aggregatesPer cluster, once every 30 secondsJMS queue for failed sensor detection
- 20. Demo of Temperature Aggregations
- 21. Event Processor for aggregationGroup by ClusterReport once every 30 secondsOver all events in the last 60 seconds
- 23. Sensor FailureWhen a sensor has not reported for 30 seconds, we consider it failedAnd potentially broken downCEP needs to trap the missing events and report those
- 24. Demo of Sensor Failure detection
- 26. Feeding Temperature Aggregate Findings into real time BAMCEP publishes result events on JMSCEP is not aware of Business Activity MonitoringBAM creates an enterprise message sourceBased on JMS and MapMessagesFeeding into a Data Object in the Active Data CacheBAM listens on the JMS Queue and maps the messages to the ADC Data Object
- 28. Java Application as Event ProducereventsComplex Event ProcessorJava applicationJMS3
- 29. Java Application producing the events to processThe user interaction with an application produces many eventsNavigation, Selection, Submission (scroll, expand/collapse, drill down)Most events are not recorded persistentlyThe events can reveal interesting information about the application, the data and the usersFor example in a web shopThe information could be used to improve the effectiveness of the application
- 30. Product DetailsSearchBuy ProductmarioMario Kart (WII)€24,992010Rating:****oDetails:1 article in shoppingbasket:€24,99Mario Kart
- 31. Super Mario
- 34. Super Mario Bros.SearchEvent‘mario’ViewEvent‘Mario Kart’BuyEvent‘Mario Kart’User interaction events in a WebShop
- 36. Derive intelligence from WebShop eventsActual Metrics: #searches in last 5 minutes, #buyers in the last 24 hours, #search-to-views drilldownsTop 10 search terms (in last [24] hour)Top 10 useless search termsAfter search on term there is no view eventMost Effective search termsSearch is followed by (one or more) view events and subsequently a buy eventItems frequently viewed but not often bought#buy/#views is ranks low
- 37. WebShop & CEPWebShop sends JMS events to single queueWhenever a user in a session searches, drills down or buysType of event, search term or item identifier, session idCEP reads from JMS channelRoutes based on event typeProcesses events using CQL processorEmits derived events to JMS queueWebShop listens to JMS queueUpdates application scope ‘top10’ beansProvides dashboard based on ‘top10’ beansPotentially using Server Push (or periodic poll)
- 38. Webshop feeding into CEP and receiving refined feedbackWebShopTop 10 Popular Search TermsTop 10 Useless Search TermsMost Effective Search Termssearchviewbuy
- 39. Popular useless search terms(missing viewevent following the search)
- 40. Real Time Event Driven Dashboard in WebShop application
- 41. SummaryComplex Event ProcessingFor analysis of real time, high volume, continuous streams of fine grained, non-persistent eventsUsed for detecting patterns, spotting exceptions, calculating aggregates and tracking trendsCEP uses CQL queries to derive information from event streams – continuously, as events continue to arriveResults are published as (higher level) events – often to JMS queues
- 42. Summary[Java] Applications can have CEP pre-process eventsto do complex analysis and filteringproducing business events that applications can understandJava Applications produce events that CEP can processAnd possibly feed back into the applicationsJMS and http are primary communication channels
- 43. ResourcesPresentation and demos are on our bloghttp://technology.amis.nl/blogContact me at:[email protected]: Chapter 19 in Oracle SOA Suite 11gHandbook describesOracle CEP and BAM
Editor's Notes
- #7: Demo in plain SQL: http://technology.amis.nl/blog/3569/simple-event-processing-using-sql-with-analytical-functions-to-detect-traffic-patterns
- #33: Wat zijn de populairste zoektermen van dit moment (bijvoorbeeld de top 10 van de afgelopen 24 uur) – dit is een vrij eenvoudige aggregatie – gegroepeerd op zoekterm en met om de zoveel uur een rapportage in de vorm van tien events voor de top zoek-termenWat zijn de populairste waardeloze zoektermen? Dat zjin zoektermen die de gebruikers kennelijk niet de door hen gewenste resultaten opleveren omdat na het zoek-event met die term niet een bekijk-event volgt, maar een nieuw zoek-event of helemaal geen event meer.Wat zijn kennelijk de meest succevolle zoektermen? Hier zoeken we naar het patroon van zoek-event gevolgd door een of meerdere bekijk–events en leidend tot een koop-event zonder dat er nog een ander zoek-event tussen zitWat zijn items die wel vaak bekeken maar niet vaak gekocht worden? Dat zijn producten waarvoor het percentage kijk+koop van het totaal aantal kijk-events laag is. Ook deze lijst kan bijvoorbeeld iedere 12 of 24 uur worden opgehoest door CEP