The IT Nerd

This really isn’t a good look if you’re Cisco.

Cisco yesterday confirmed a ransomware group known as Yanluowang breached their network in May, resulting in the group attempting to extort Cisco under the threat of leaking stolen files. Cisco has since revealed that the attackers only harvested and stole non-sensitive data from a Box folder linked to a compromised employee’s account. I am not sure if I buy that. But that’s the information that is out there from Cisco. Speaking of information, if you want to really go into the weeds, Cisco has additional information here that’s very much worth reading.

Sharon Nachshony, Security Researcher for Silverfort had this to say:

    “The activity seen in the Cisco attack is a prime example of how an attacker can use lateral movement to progress from an initial toehold towards more high-risk internal targets.

Starting with a single set of stolen credentials the attacker was able to gain access into the Cisco VPN, pivot into the Citrix environment and eventually move to the domain controllers. Their use of PsExec in the attack was notable. Command line tools such as this are typically used by admins to remotely configure and troubleshoot – but in the hands of an attacker, and often unprotected, they have become a target of choice. This can be prevented by applying MFA to remote command tools to manage access and close down lateral movement.

More broadly, this is a sign of how lateral movement is being commoditized by initial access brokers. Focusing specifically on initial breach and accessing of target systems, they will then sell this compromised position on to other threat actors specializing in payloads and ransom activity.

Given that who Cisco is, this is pretty embarrassing. The damage to their reputation is going to be significant. Which is part of the reason why companies pay threat actors (even though they shouldn’t) as they don’t want the damage to their reputation. While I do give Cisco credit for coming clean about this, they should not be in this position in the first place. But I guess it proves that any company can be pwned.

UPDATE: I have additional commentary from two sources.

Mike Pedrick, VP, Cybersecurity Consulting at Nuspire had this to say:

“As details emerge regarding the sequence of events that lead to the breach of sensitive data from industry giant Cisco, it becomes clear that our employees are still our most vulnerable – and most often targeted – assets.  By all appearances, Cisco was doing the right things by implementing VPN technologies and leveraging multifactor authentication for access.  This is the baseline standard for prudent behavior on the part of any business and even the most paranoid of security consultants would nod their heads gently – if begrudgingly for some – in approval of the protocol.

But in this case, social engineering won out again.  After obtaining the victim’s credentials by compromising an unrelated system, attackers bombarded the victim’s mobile device with MFA Push notifications in the hopes that the victim would approve the authentication request.  Spoiler alert: they did.

If there’s a lesson to be learned from this latest breach, it’s similar to the last one and the one before that.  All organizations are at risk and attackers aren’t pulling their punches. Continue doing the right things and hope for the best but prepare for the worst and when things do go wrong, you’ll be more ready than if you assume your defenses – and people – are perfect.”

Keatron Evans, principal security research at Infosec Institute also had this to say:

“This is yet another example of no matter what controls you have in place, an end user having even a slight temporary lapse in judgement can bring the entire security palace crumbling down. From what I can deduce right now, Cisco had most of the right things in place; End user awareness training? Check.  Multi-factor Authentication? Check. Robust endpoint detection and response? Check.  But they were still compromised. This may also be one of the largest breaches involving vishing and smishing.  As users combine more of their personal lives with work lives, especially technologically, we will see more of these attacks whereas it starts with the victim’s personal credentials and then leads to compromise of their corporate credentials. 

Questions which have not been answered as of yet:

• What type of data was taken? I ask this because 2.8 Gigabytes of internal training videos likely less significant than 2.8 Gigabytes of proprietary Cisco source code. 
• If this happened in or before May and Cisco found out in May, why did disclosure take so long? 
• It has been stated multiple times that the TTPs or tactics, techniques, and procedures, overlap with a couple of other threat actor groups, such as Lapsus$; As hackers/threat actors we “borrow” TTPs all the time if they’re effective. How sure is Cisco about these specific attributions and associations?
• Although Cisco has made clear they feel they contained the threat and eradicated it, and blocked many attempts the threat actors made to get back in. Has there been any post incident threat hunting activities conducted based on the threat hypothesis that the threat actor group is still inside and moving horizontally?
• How was Cisco alerted to the breach? 
• Was this end-user working from home? Or in the office?  If at home, would this attack have been less successful or prevented if the user were working from the office?”

Cisco AppDynamics, a leading provider of Observability and Application Performance Monitoring technology, has published findings from Agents of Transformation 2022, the fourth annual report that analyzes the skills and attributes of elite global technologists.

In the wake of the pandemic, it reveals the emergence of a new class of technology experts stepping up to meet critical challenges that are blurring the lines between business strategy and IT operations. The report also cites the demand to make all products and services digitally available in the Experience Economy amid heightened security threats, increasing complexity, and the accelerated shift to hybrid work and the cloud.

According to the Cisco AppDynamics report, 74% believe that their experiences in recent years—particularly during the pandemic—have accelerated their careers, and 88% now consider themselves to be business leaders. However, just 10% of technology experts have reached the elite status of ‘Agents of Transformation’. These individuals represent top-flight leaders who are reimagining and delivering high-value applications and services that create the always-on, secure, and exceptional user experiences now demanded by end users and customers.

Respondents cite a fundamental change in the role of technologists, including the skills and resources required to operate effectively and proficiently. At the same time, they say they now contend with soaring complexity and volumes of data from across the technology stack and must integrate a massively expanding set of cloud-native services with existing on-premises systems and tools.

• 88% believe that what it means to be a technologist has changed 
• 84% say the skills and qualities that define an Agent of Transformation have evolved
• 66% indicate that it is now more difficult to be an Agent of Transformation
• One in four say their organization remains stuck in reactive, “fire-fighting mode”

Digital transformation means almost every company and organization interacts with consumers via web and mobile applications, and the transition to hybrid work means more interaction with SaaS tools and web interfaces. While consumers can pivot fast to another brand’s app or service, companies that cannot instantly improve digital experiences risk having loyal customers walk away.

While acknowledging the far-reaching consequences of this change, respondents in the Cisco AppDynamics report note that they need help navigating the technical and operational ambiguities of digital transformation. Specifically, they are looking for unified visibility into their IT environments to better manage and optimize application availability and performance. This requires focusing investments on application security, observability over cloud-native applications and infrastructure, and linking IT performance to business decision making. 

• 77% believe it will be important to invest in application security over the next 12 months to meet customer and employee needs
• 71% think their organization will need to invest in observing cloud-native applications and infrastructure
• 84% say that the need to maintain the performance of business applications is now more important than ever
• 85% state that full stack observability is core to sustainable transformation and innovation in their organization

Additional Resources: 

Download the Agents of Transformation 2022 report: https://www.appdynamics.com/resources/reports/agents-of-transformation-2022

Read more on the AppDynamics blog: https://www.appdynamics.com/blog/news/agents-of-transformation-are-adapting-at-speed-to-drive-innovation-in-the-experience-economy

Today, Cisco announced the launch of AppDynamics Cloud at Cisco Live, the premiere networking and security event. AppDynamics Cloud enables delivery of exceptional digital experiences by correlating telemetry data from across any cloud environment at massive scale. It leverages cloud-native observability to remediate application performance issues with business context and insights-driven actions.

AppDynamics Cloud maximizes business outcomes and customer experiences by continuously optimizing cloud-native applications. It accelerates detection and resolution of performance issues, before they impact the business or the brand, with intelligent operations. Investment protection is derived from continuous data integrations with OpenTelemetry ™ standards and technology partnerships with cloud solutions and providers.

The platform enables collaboration across teams including DevOps, site reliability engineers (SREs), and other key business stakeholders to achieve common benchmarks like service-level objectives (SLOs) and organizational KPIs. While many organizations still run their mission-critical and revenue-generating systems with traditional applications, modern business apps are increasingly built using DevOps initiatives and must support distributed architectures and services. This pandemic-accelerated trend has spawned an end-to-end experience revolution among consumers and end users, and hybrid work is contributing exponential momentum.

To deliver the consistent, reliable digital experiences that consumers and end users now demand, IT teams must monitor and manage a dynamic set of application dependencies across a mix of infrastructure, microservices, containers, and APIs using home-grown IT stacks, multiple clouds, SaaS services, and security solutions. Traditional monitoring approaches break down in this vastly complex and dynamic ecosystem.

AppDynamics Cloud seamlessly ingests the deluge of metrics, events, logs, and traces (MELT) generated in this environment—including network, databases, storage, containers, security, and cloud services—to make sense of the current state of the entire IT stack all the way to the end user. Actions can then be taken to optimize costs, maximize transaction revenue, and secure user and organizational data.

Current AppDynamics customers can upgrade to AppDynamics Cloud and leverage their existing application performance monitoring (APM) agents, or feed both solutions concurrently. AppDynamics Cloud supports cloud-native, managed Kubernetes environments on Amazon Web Services (AWS), with future expansion to Microsoft Azure, Google Cloud Platform, and other cloud providers.

The pandemic put a pause on travel over the last few years, so it’s no surprise consumers are eager to make up for lost time. In fact, 76% of Canadians say they are looking forward to taking a trip this year. But as they plan their much-anticipated summer vacations, just how dependent they’ve become on digital applications to make it happen might surprise you.

From researching destinations to scheduling flights and booking hotels, the average Canadian uses 23 different digital applications throughout their vacation process. It’s clear that digital applications and services have become a crucial part of the vacation experience, which means the pressure is on for brands to ensure their apps perform flawlessly this summer.

In its new global study of consumers, including Canadians, Cisco AppDynamics explored how applications are being used throughout the vacation process and the significance of their role today.

Key insights from Canadian travelers:

• 64% say applications and digital services are now central to their vacation experience
• 70% say if a travel app, like an airline or hotel booking, were to fail it would disrupt their vacation and 42% claim it could ruin their whole trip
• 33% say if they experience a problem with an application when planning, booking or traveling on vacation, they’ll immediately switch to an alternative
• 42% say they don’t intend to carry any paper-based tickets, relying entirely on applications such as digital wallets throughout their trip

You can have a look here for additional details from the study and tips on how technologists can leverage full-stack observability to maintain the performance of their applications this summer.

There’s been substantial growth in the consumer medical devices market in recent years – 320 million consumer medical wearables will ship globally in 2022 (according to Deloitte). These range from heart rate monitors that can be used to detect heart disease and long COVID, to bracelets which aid ovulation prediction and conception. Now, consumers are incorporating this technology in their daily lives to improve their overall health and wellbeing.  

In a new study of more than 12,000 consumers globally, including Canada, Cisco AppDynamics uncovered how quickly consumers are adopting this technology, the level of trust they have when allowing third parties to handle their data, and their expectations for incredible digital experiences when using these services.  

The results show a booming industry, with consumers keen to realize a range of health and wellbeing benefits. But at the same time their expectations for flawless digital experiences are higher than ever. One bad digital experience could be the make-or-break moment in a technology failing to reach its full potential. 

Key Canadian takeaways from the report include: 

• 2% of Canadians think wearable technology has the potential to transform both their personal health and public health as a whole 
• 61% of Canadians say they intend to use more of these types of wearable technologies or applications in the next 12 months 
• 33% of Canadians say they currently use at least one wearable health tech device 
• 73% of Canadians say a bad digital experience may stop them using a specific wearable device or application and 51% say it may put them off trying other health or wellbeing wearables or applications 

• The biggest components of a bad digital experience for Canadians are: 
  • Data privacy / data security leak (61%) 
  • Application or device crashing (58%) 
  • Slow run time / unresponsive (57%) 

• 86% of Canadians say reliable, real-time access to health data and accuracy of this data is critical to a good user experience 

There is a lot more detail on this report which you can find here.

If you use Cisco WebEx to meet with people, you should be aware that it will phone home audio telemetry according to some research performed on the most popular conferencing apps out there and reported by The Register. And muting the app has zero effect on this:

Among the apps studied — Zoom (Enterprise), Slack, Microsoft Teams/Skype, Cisco Webex, Google Meet, BlueJeans, WhereBy, GoToMeeting, Jitsi Meet, and Discord — most presented only limited or theoretical privacy concerns. The researchers found that all of these apps had the ability to capture audio when the mic is muted but most did not take advantage of this capability. One, however, was found to be taking measurements from audio signals even when the mic was supposedly off. “We discovered that all of the apps in our study could actively query (i.e., retrieve raw audio) the microphone when the user is muted,” the paper says. “Interestingly, in both Windows and macOS, we found that Cisco Webex queries the microphone regardless of the status of the mute button.” They found that Webex, every minute or so, sends network packets “containing audio-derived telemetry data to its servers, even when the microphone was muted.” 

This telemetry data is not recorded sound but an audio-derived value that corresponds with the volume level of background activities. Nonetheless, the data proved sufficient for the researchers to construct an 82 per cent accurate background activity classifier to analyze the transmission and infer the likely activity among six possibilities — e.g. cooking, cleaning, typing, etc. — in the room where the app is active. Worse still from a security standpoint, while other apps encrypted their outgoing data stream before sending it to the operating system’s socket interface, Webex did not. “Only in Webex were we able to intercept plaintext immediately before it is passed to the Windows network socket API,” the paper says, noting that the app’s monitoring behavior is inconsistent with the Webex privacy policy. The app’s privacy policy states Cisco Webex Meetings does not “monitor or interfere with you your [sic] meeting traffic or content.”

Well, clearly what is in their privacy policy is at best inconsistent with what they actually do. And at worst it’s a lie. But don’t worry, Cisco “fixed” this after it was pointed out to them:

Cisco told The Register that it altered Webex after the researchers got in touch so that it no longer transmits microphone telemetry data.

“Cisco is aware of this report, and thanks the researchers for notifying us about their research,” said a Cisco spokesperson. “Webex uses microphone telemetry data to tell a user they are muted, referred to as the ‘mute notification’ feature. Cisco takes the security of its products very seriously, and this is not a vulnerability in Webex.”

No it’s not a vulnerability. But it’s pretty bad from an optics standpoint and from a trust standpoint. Hopefully they don’t have anything else in their products that someone can trip over and call them out on. Because that’s won’t end well from a PR standpoint.

After two years of rapid digital transformation and firefighting in response to the pandemic, technologists are now ready and primed to drive the next critical wave of innovation in their organizations. To achieve their goals, IT teams are seeking out solutions which will give them full visibility across their IT environment.

They’re looking for a unified view on availability and performance up and down the IT stack, building on their current application monitoring tools. They want full visibility into performance for compute, storage, network and public internet, from the customer-facing application all the way into the back end. For many, they want solutions that provide full visibility into cloud native environments, including the increasing deployment of microservices and container solutions.

In a new study of 1,200 IT professionals worldwide, including Canada, Cisco AppDynamics found that appetite for full-stack observability has markedly increased over the past year, and an overwhelming majority of Canadian IT professionals (92%) said 2022 will be a pivotal year for their organization on the journey towards full-stack observability as they look to unlock the power of data in an application-driven world. 

Other key takeaways from Canadian IT professionals in the report include:

• 91% of Canadian IT professionals believe the shift to full-stack observability will be transformational for their business.
• More than half of Canadian organizations (60%) have started out on the journey to full-stack observability, and a further 35% are planning to do so in the next 12 months.
• 86% of Canadian IT professionals accept that organizations that fail to make significant strides in their journey towards full-stack observability in 2022 will face competitive disadvantage versus their peers. 

A full copy of AppDynamics’ Journey to Observability report can be found here.

Research Methodology

To gauge the extent to which technologists have succeeded in implementing their full-stack observability strategies, and to understand their priorities for the future, AppDynamics has undertaken comprehensive global research, from board-level directors and CIOs, through to senior and mid-level IT management.

This research entailed:

• Interviews with 1,200 IT professionals in organizations with a turnover of at least 500m (with the exception of Colombia, where organizations with a turnover of at least $100m were included in the sample.)
• Interviews were conducted in 14 markets – Australia, Brazil, Canada, Colombia, France, Germany, India, Japan, Mexico, Russia, Singapore, United Arab Emirates, United Kingdom and United States.
• Respondents worked across a range of industries, including IT, financial services, retail, public sector, manufacturing and automotive, and media and communications.
• All research was conducted by Insight Avenue in December 2021 and January 2022.

Uses of Cisco Firepower firewalls need to pay attention to this.

Cisco has put out a Field Notice advised that the SSL certificate authority used to sign certificates for Talos security intelligence updates will be decommissioned and replaced on March 6, 2022. What that means is that Firepower devices “might” not be able to receive Talos updates. Those updates contain lists of sites identified as sources of malware, spam, botnets, and phishing to these firewalls. The firewalls in turn can automatically apply them so that you don’t have to add to the always-growing list of threats manually. So in short, after March 6, you might not be protected from the latest threats that exist on the Interwebs. Which of course is bad.

Users of FirePOWER Services Software for ASA, Firepower Threat Defense (FTD) Software, Firepower Management Center Software, and Firepower 6.1.x through 7.1.x have therefore been advised they’ll need to update their software. The update is required for both physical firewalls and FirePOWER running in clouds. And the deadline for doing the update is March 5th. Which means that you don’t have a lot of time to do these updates which are already available. The only exception is those who run Firepower 7.1.x, who have been warned that their update is “Planned for release by March 1, 2022.” That’s four days which is an insanely short amount of time.

Thus you should prepare to get about patching all the things. Because you know that cybercriminals will be getting ready to pwn those who don’t patch all the things.

By Gregg Ostrowski

Last year in Canada, the Cyber Centre had knowledge of 235 ransomware incidents against Canadian victims from 1 January to 16 November 2021. And the estimated average cost of a data breach, a compromise that includes but is not limited to ransomware, is $6.35M CAD. This presents a massive challenge for organizations to better safeguard their data and the data of their customers. For many, applications and digital services became a lifeline to normality but at the same time their expectations for applications to be high-performing, reliable and secure sky-rocketed.

Launched in 2006 by the European Council to raise awareness around the rights to personal data protection and privacy, Data Privacy Day encourages individuals and organizations to respect privacy, safeguard data and enable trust. According to Gregg Ostrowski, Executive CTO at Cisco AppDynamics, the need for everyone to think and act carefully has never been greater when it comes to sharing and protecting our personal data.

“The AppDynamics App Attention Index 2021 showed that for consumers, security is the number one component of a high performing ‘total application experience’. And 73% of Canadians say that their expectation of brands to keep their data secure has increased since 2020. It goes to show that brands must go above and beyond to meet their users’ expectations towards security,” said Gregg Ostrowski.

“In this post-pandemic era, a strong security posture means organizations have the necessary processes in place to protect their applications and their business from vulnerabilities and threats. In a world where sensitive data is constantly at risk of being compromised by malicious actors, they must be prepared and strengthen their security posture, enabling them to predict, prevent and respond to threats.”

Gregg also says it’s important for organizations to ensure security is at the core of their applications development process. “The DevSecOps methodology, a modern approach to software development, takes things a step further and incorporates security enhancements at the beginning of the application development lifecycle for a more proactive approach to reduce risks of threats to sensitive customer data.”

However, according to Gregg, there’s another piece to the puzzle for a DevSecOps approach to be fully effective. “Teams need to implement a full-stack observability solution. This approach will give them in-depth visibility into the entire IT stack, including traditional legacy systems through to new, native cloud environments as well as hybrid deployments. It is a vital step in the right direction.”

About Gregg Ostrowski

Gregg Ostrowski is an Executive CTO at Cisco AppDynamics. He engages with customer senior leadership to help prioritize their strategy for digital transformation. Prior to AppDynamics, Gregg held senior leadership positions at Samsung and Research in Motion, and he has more than 20 years of experience in the industry.

A recent high profile digital service outage saw Facebook – including its other services/brands Instagram, WhatsApp and Oculus – offline for over 6 hours, causing widespread disruption for consumers that use these applications and digital services on a daily basis, as well as businesses that rely them as part of their operations. 

But Facebook is not alone in facing this type of crisis.

In a world of sky-high expectations for digital experiences, and record levels of intolerance for poor-performing applications, this level of disruption can cause businesses significant financial cost and damage to reputation.   

In the immediate aftermath of the outage AppDynamics surveyed 1,011 global IT decision makers in 11 countries to gauge their concern surrounding the impact of major outages, understand the pressure they are facing and their confidence in the technology, tools and processes they are currently using. The results are as follows:

• 87% of enterprise technologists say they are concerned about the potential for a major outage and the resulting disruption to their applications and digital services. 
• 84% say that they feel pressure from their organization’s leadership to prevent a major performance issue or outage of their customer and employee facing applications and digital services.
• 87% admit that increasing complexity of their IT stack is already causing delays in identifying the root cause of issues.
• 97% of IT teams have some form of monitoring tools in place, many of which provide highly sophisticated and advanced methods of identifying and fixing anomalies. But they question the effectiveness of these tools in this new world. Only 27% are entirely confident that they meet their needs.
• 72% think it is critical or important that their organization deploys a full-stack observability solution within the next 12 months to solve complexity across their IT stack and identify and fix the root cause of an issue.

There is more information in this AppDynamics blog post here: https://www.appdynamics.com/blog/news/businesses-fear-outages-full-stack-observability/.

Research conducted between 7th – 11th October 2021