Connect with us

Business

Why success brings its own challenges in IT: Fixing the real-time problems of real-time event streaming with an event portal

Spokesperson: Tom Fairbairn, Distinguished Engineer, Solace

Amidst the recent hype around generative AI, there is a perhaps quieter technology revolution going on: the increasing adoption of event-driven architecture (EDA), an example of which is “event streaming” across the enterprise. Regardless of industry, (manufacturing, retail or even HR, amongst others), adoption of EDA is increasing use cases across the board as organisations realise the value of EDA. From decoupling applications, the ability to be more responsive to internal and external user requests, and reducing operational costs, EDA ultimately allows enterprises the freedom to become more agile and real-time in their day-to-day activities.

A recent IDC Infobrief found that of those who have deployed EDA across multiple use cases, 93% say it either met or exceeded their expectations. Going even further, 82% of those surveyed have stated their plans to apply EDA across their company for a further two/three use cases, within the next 24 months.

Increasing adoption and time-to-market scrambles, along with bigger data flows and more complex deployments lead to difficulties in getting real visibility across a complex, and often rapidly scaling ecosystem of events.

As a result, some are struggling to realise the best return on their EDA investment. The issue can sometimes compound, as an organisation may have reached a stage of maturity regarding multiple streaming use cases, but are left with siloed brokers, clusters, topics and schematics weaved on top of each other. A layer cake effect occurs in the architecture, making it difficult to truly see where they can optimise the information insights chain.

Growing pains are a natural part of scalability, and EDA is no exception.

Three pain points typically arise:

1-Poor visibility requires a clear and clean portal

One of the unsung heroes of EDA is application decoupling, which aids flexibility and agility. However, this can often be detrimental to visibility, making changes to existing applications difficult and risky. This lack of visibility leads to confused ownership, as event producers don’t know who the consumers are, and consumers don’t know who the publishers are. Some event streaming systems, such as Kafka, rely on static or near-static topics which can be abandoned or duplicated.

This directly affects the end-user experience. Say a person is using the application every day and wants to add an extra attribute to the data – they’d have no direct line to contact because they can’t see who produced the data.

An event portal’s single window into the structure of an EDA benefits the event-streaming ecosystem by acting as a native discovery agent to quickly produce visual aids to help locate consumers and producers. Architects and administrators on the business side are able to view the relationship between producers and consumers, as well as their event interface and KPIs, including their most and least-used topics.

Another issue that occurs when scaling infrastructure and information flows is the inability to fully grasp the downstream impact of changes made to the architecture. Evolving and architecture will require visibility on whether microservices are affected by a given event before even deploying a new feature or function, to ensure it doesn’t bring down that system. This must be done sometimes in seconds. While real-time is part of the solution, an event portal can automate the scanning of an internal system, helping visualise a complete view of endpoints and event streams, rather than making the business manually draw them between microservices. For those looking to better understand their infrastructure, particularly as they scale up, an event portal helps deploy new services faster.

2-Event Streaming is not a one-and-done – the issue with limited sharing and reuse

Data has been the new oil for quite some time, and much like oil, when one needs to see data, they often need it right there and then. Real-time data is the most valuable data that can exist within your organisation, but if it is siloed in one specific department, overall business decision-makers don’t know about it, and that means it is not providing its full potential.

Developers must understand that their hard work won’t be realised until they can catalogue all that data and the benefits that come with it. Otherwise, it may get stale and quickly become out of date.

An event portal can create a real-time catalogue of event data, able to list all topics, event streams and schemas for each application, as well as the owner and best point of contact. Not only does this speed up internal developments by letting developers share, discover, and re-use existing event-streaming assets, but they can do so outside of the organisation so that customers and partners can benefit from it too – an often neglected consideration when technologies such as Kafka focus on operational management.

3-Security and governance in event streams

The fact that event-driven systems are dynamic and decentralised is a major selling point. However, that can run the risk of unique security challenges.

A trade-off that commonly occurs when implementing event streaming is that, as they include access control rules, developers tend to be too permissive to ensure agility, creating a lack of visibility or proper cataloguing of the data. With time, that runs the risk of interfering with business data security, governance and compliance. The issue only gets worse over time as the use of data evolves across the applications, with each new human addition.

With a thinning attack surface and an increase in never-before-seen malware variants, security in business must pivot from reactive to proactive.

An event portal helps users organise systems into application domains, create and import payload schema definitions in a variety of formats and better define event interactions. This improves IT governance by more easily allowing control for who can access which resources while giving those the ability to create and track every version of EDA objects as they evolve.

By putting visibility at the forefront, administrators can spend less time worrying about ensuring their security is in line with internal and regulatory policies. Outside of real-time visibility, the true value of event-driven architecture lies in the ability to easily spot and re-use all of these data assets. However, that can only be done if they are documented, managed and governed properly. The event portal is necessary to solve this, acting as a discovery tool, de facto guide, and being able to dictate the more effective way to manage event streams.

Real-time event streams and the success of powerful architectures have brought with them the need to address the rapidly growing complexity of event streaming estates. Single, multi-broker event portal technology is the solution to helping organisations discover, document, govern and manage the lifecycle of their real-time event streams across the enterprise, and it will become increasingly important as more organisations embrace EDA as a foundational platform.

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Business

Building Compliance into Business Culture is Essential in Fintech

Source: Finance Derivative

Tetyana Golovata, Head of Regulatory Compliance at IFX Payments

Regulation plays a critical role in shaping the fintech landscape. From Consumer Duty and FCA annual risk reporting to APP fraud, the tectonic plates of the sector are shifting and whether you consider these regulations as benefiting or hindering the industry, businesses are struggling to keep up. 

According to research by fraud prevention fintech Alloy, 93% of respondents said they found it challenging to meet compliance requirements, while in a new study by Davies a third of financial leaders (36%) said their firms had been penalised for compliance breaches in the year to June. With the FCA bringing in its operational resilience rules next March, it is more important than ever to ensure your company makes the grade on compliance. 

Lessons from history

Traditionally, FX has struggled with the challenge of reporting in an ever-developing sector. As regulatory bodies catch up and raise the bar on compliance, responsible providers must help the industry navigate the changes and upcoming deadlines.

Fintechs and payments companies are entering uncharted waters – facing pressure to beat rivals by offering more innovative products. When regulators have struggled to keep up in the past, gaps in legislation haveallowed some opportunists to slip between the net, as seen in the collapse of FTX. Because of this, implementation and standardisation of the rules is necessary to ensure that innovation remains seen as a force for good, and to help identify and stamp out illegal activity.

Culture vs business

Culture has become a prominent factor in regulatory news, with cases of large fines and public censure relating to cultural issues. As the FCA’s COO Emily Shepperd, shrewdly observed in a speech to the finance industry, “Culture is what you do when no one is looking”.

Top-level commitment is crucial when it comes to organisational culture. Conduct and culture are closely intertwined, and culture is not merely a tick-box exercise. It is not defined by perks like snack bars or Friday pizzas; rather, it should be demonstrated in every aspect of the organisation, including processes, people, counterparties, and third parties.

In recent years, regulatory focus has shifted from ethics to culture, recognising its crucial role in building market reputation, ensuring compliance with rules and regulations, boosting client confidence, and retaining employees. The evolving regulatory landscape has significantly impacted e-money and payments firms, with regulations strengthening each year. Each regulation carries elements of culture, as seen in:

  • Consumer duty: How do we treat our customers?
  • Operational resilience: How can we recover and prevent disruptions to our customers?
  • APP fraud: How do we protect our customers?

Key drivers of culture include implementing policies on remuneration, conflicts of interest, and whistleblowing, but for it to become embedded it must touch employees at every level.

This is showcased by senior stakeholders and heads of departments facilitating close relationships with colleagues across a company’s Sales, Operations, Tech and Product teams to build a collaborative environment. 

Finance firms must recognise the trust bestowed on them by their customers and ensure the protection of their investments and data is paramount. Consumer Duty may have been a wake-up call for some companies, but progressive regulation must always be embraced and their requirements seen as a baseline rather than a hurdle.

Similarly, the strengthening of operational resilience rules and the upcoming APP fraud regulation in October are to be welcomed, increasing transparency for customers. 

Compliance vs business 

Following regulatory laws is often viewed as a financial and resource drain, but without proper compliance, companies are vulnerable to situations where vast amounts of money can be lost quickly.

A case in point is the proposed reimbursal requirement for APP fraud, which will mean payment firms could face having to pay compensation of up to £415,000 per case.

Complying not only safeguards the client and their money, but also the business itself. About nine in ten (88%) financial services firms have reported an increased compliance cost over the past five years, according to research from SteelEye.  Embedding compliance earlier in business cultures can be beneficial in the long run, cutting the time and money needed to adapt to new regulations and preventing the stress of having to make wholesale changes rapidly. 

Building a cross-business compliance culture 

Compliance is a key principle at IFX, and we strive to be a champion in this area. In response to these challenges, the business restructured, establishing dedicated risk and regulatory departments, along with an internal audit function. 

Regulatory compliance aims to support innovation by developing and using new tools, standards, and approaches to foster innovation and ensure product safety, efficacy, and quality. It has helped the firm to navigate the regulatory landscape while driving growth and maintaining high standards.

This organisational shift allowed each business line to own its own risk, with department partaking in tailored workshops designed to identify existing, new, and potential risk exposure. Shared responsibility for compliance is the only way to create a culture which values it. We see this as a great way for organisations to drive innovation while sticking to the rules. 

Continue Reading

Business

How AI virtual assistants are transforming education and training

By Gregor Hofer, CEO and Co-founder at Rapport

What separates good doctors from excellent doctors, the type that might get five-star reviews if, like an Uber driver, their services were supported by a smartphone app?

Medical knowledge, expertise, and better outcomes are, of course, the most important factors. But – particularly when dealing with patients’ relatives, discussing risk assessment and imparting bad news – we shouldn’t underestimate the importance of bedside manner.

This might come naturally to some doctors but there are none for whom training isn’t useful, whether at medical school or on the job.

There will always be a place for real human interaction in this training, the type that involves role-play, with actors or colleagues playing out different scenarios that explore the most effective ways to handle difficult situations.

But what if this could be supplemented by more readily available and less resource-intensive experiences that simulate these training environments? And what if it could be applied across numerous sectors, industries and professions, of which there are a great many that could benefit from such an opportunity?

What might that mean for those instigating tricky conversations and, perhaps more importantly, those at the receiving end of them?

Advances in generative artificial intelligence – or GenAI – mean that these are no longer hypothetical questions.

There’s no limit to the type of person this technology could help, but we’ll review three – doctors, those working in corporate HR, and online students – to give a flavour of the benefits it brings.

Before we do, a quick word on how such applications work.

An overview of the technology

It all starts with data. With access to enough content, the type that you store and curate on your internal systems, large language models (LLMs) can be trained to find the most appropriate response to whatever user input they’re exposed to, whether in writing or spoken, and then you as a user can respond to that response, and so the cycle continues.

You’ll have experienced something similar using the likes of CharGPT, but because this is based on your own content, you’re more in control. (For simpler and more prescriptive scenarios, though, I’d add that with the best solutions, you can alternatively import predefined branching dialogue to keep your conversations on track.)

It doesn’t stop there, though; by tapping into a solution that’s supported by experts in linguistics and computer-aided animation, your colleagues can interact in real-time with avatars equipped with believable facial expressions, accurate lip-synching capabilities, natural gestures and the ability to detect emotions.

All of this adds to the user’s willing suspension of disbelief that they’re interacting with a real person, or AI avatar, thereby enhancing the effectiveness of their learning.

These innovations are reshaping how we approach learning and skill development in so many critical fields. We said we’d look at three. We’ll start by returning to medicine.

Medical training

AI assistants can supplement the way doctors are taught to break bad news to patients, one of the hardest things they’ll face in practice and, given its subjectivity, something that can’t easily be looked up in a textbook on anatomy or physiology.

As we said from the outset, this is easier for some doctors than others, but given the literal life-and-death nature of such conversations and the shattering impact that the death of a loved one can have on a relative, there’s always room to improve medics’ empathy and communication skills – which is exactly what this technology delivers.

By utilizing experiential AI tools, clinicians can better use their time, alleviate pressure, fatigue and burnout symptoms, and ultimately allow them to better serve their patients.

Corporate HR

In corporate HR, virtual assistants can significantly streamline and enhance the hiring and firing process, as well as any difficult conversation; whether it’s a tough review, a disciplinary hearing, letting down an employee about a promotion they’d applied for or any other scenario that might bring a bead of sweat to your forehead, it’s all about providing safe and cost-effective practice before doing it for real.

Tech research consulting firm Gartner recently found that more than three-quarters (76%) of HR leaders believe that if their organisation doesn’t adopt and implement AI solutions, such as generative AI, in the next 12 to 24 months, they’ll lag in organizational success compared to those that do, while 34% of HR leaders participating in their January benchmarking session said they were exploring potential use cases and opportunities when it came to generative AI.

If they do manage to adopt the right technology, the impact will be massive among those who deploy it wisely. After all, which company wouldn’t want to upskill its HR professionals in tangible soft skills such as empathy, communication, problem-solving, and conflict resolution in a controlled setting?

Online education

AI-powered tools can hugely boost student engagement in remote learning environments, and the research suggests that it comes close to rivalling in-person experiences. When you consider the staff-to-student ratios common in most educational settings, this should be no surprise – think how many students can fit into a lecture hall (even if they don’t always turn up!).

But we’re not necessarily talking about formal education; this applies equally to any informal setting in which someone needs to improve their education in some way.

With this technology, you can invent new ways to educate your students – or staff – by transforming lessons into experiences, using interactive characters reflective of the subject. This means you can increase user satisfaction and performance without compromising on content.

Whatever the scenario and whatever the use case, the chances are that if you have the right content in sufficient quantities, you can tap it for interactions that would otherwise be lacking in uniqueness or prohibitively expensive.

With AI virtual assistants, everyone’s a winner.

Continue Reading

Business

How GenAI is Shaping the Future of Compliance

Gabe Hopkins, Chief Product Officer, Ripjar

Generative AI or GenAI uses complex algorithms to create content, including imagery, music, text, and video with amazing results. Less well known are some of the ways in which it can transform data processing and task performance. This groundbreaking technology not only saves time, effort, and money, but has become a game-changer in enhancing operational efficiency and fostering innovation across various sectors.

However, some industries like anti-financial crime compliance – have been slow to adopt new innovations like GenAI, predominantly due to concerns over potential risks. In fact, they can even see it as a risk in itself. Legal, Compliance and Privacy leaders rank rapid GenAI adoption as their top issue in the next two years, all while other, less risk-averse organisations enjoy the upside of implementing GenAI in their systems.

This delay means many compliance teams are not taking advantage of AI tools that could revolutionise their processes and help them save up to 200 hours annually per user.

Entering the New Era of GenAI in Compliance

Teams in largely regulated sectors like banking and fintech face enormous pressures. Their responsibilities include identifying risks, such as sanctioned individuals and entities, updating policies to keep up with ever-evolving regulations, and handling expansive datasets. The high volume of this data makes manual reviews exhausting and susceptible to errors, which can lead to financial and reputational damage.

One way to overcome these challenges is by leveraging GenAI. For example, false positives – where a risk is raised incorrectly or false negatives, where a real risk is not flagged, are common issues caused by trying to deal with very high volumes of alerts and risk matches. Implementing GenAI can reduce these inaccuracies, significantly enhancing the efficiency and effectiveness of customer and counter-party screenings.

In practical terms, GenAI can reinvent how compliance tasks are performed. For instance, in drafting Suspicious Activity Report (SAR) narratives, where analysts need to justify suspicions in transactions, GenAI can help automate this writing process, combining human oversight with artificial efficiency. Platforms using GenAI excel in summarising vast amounts of data— crucial for tasks like screening adverse media, where they assist in identifying potential risks linked to negative information about clients.

 Understanding the Opportunities of GenAI and Overcoming Fears

For the compliance sector, it’s a crucial time to explore how to incorporate GenAI effectively and securely without undue risks. Dispelling fears about data misuse, the high costs of initial model setups, and the ‘black box’ nature of AI models are central to this transition. Teams are particularly cautious about sharing sensitive data and the hidden biases that AI might carry.

Yet, some strategies can counter these challenges. By choosing suitable models that ensure robust security and privacy and adjusting these models within a solid statistical framework, biases can be mitigated. However, organisations will need to turn to external expertise – whether data scientists or qualified vendors – to support them in training and correctly deploying AI tools.

The latest advancements in GenAI suggest that virtual analysts powered by this technology are achieving, and sometimes surpassing, human-level accuracy. Despite ongoing concerns, which may slow adoption rates, the evident potential benefits suggest a bright future for compliance teams using GenAI. These technological innovations promise not only to improve speed and efficiency but also to enhance the capability of teams to respond and adapt swiftly.

Embracing GenAI will not only significantly elevate the effectiveness of compliance operations but also safeguard organisations against potential pitfalls while maintaining trust and integrity in their industry practices.

Continue Reading

Copyright © 2021 Futures Parity.