Monitoring Cloud-based Applications Best Practices

Posted on by

What is your cloud monitoring strategy? After all, a whopping 93% of organizations in 2015 are running applications in the cloud or experimenting with infrastructure-as-a-service, while 82% of enterprises have a hybrid cloud strategy, up from 74% in 2014 (Source: RightScale 2015 survey).
Cloud pic

As you grow, and join the ranks of these enterprises moving to the cloud, or leveraging cloud-based applications, there is one thing that you should not forget. Regardless of where your applications are hosted, your users will hold IT responsible for their user experience! To ensure smooth operations and user satisfaction, you need to monitor cloud-based applications from a user perspective, just like any other application in your portfolio. Here are some quick tips to jump start your cloud monitoring efforts.
 
1. Prioritize your cloud-based applications. With an increasing number of employees bypassing IT and going rogue to the cloud, it is the wild, wild, west out there.  From accounting, invoicing, HR, CRM, email or collaboration tools, the list of applications your employees rely on is long! Since you can’t monitor it all at once, identify those critical applications that must perform well in order to run your business (e.g. internal applications migrated to the cloud, cloud-based CRM, ERP, EHR systems), and monitor these cloud-based applications first.

2. Identify critical transactions to monitor. Put on your “think from a user perspective hat” and map out common functions used by your power users (those driving the most productivity or revenue). Or better yet, schedule a meeting with business stakeholders to identify critical functionality from their perspective. For example, if you are monitoring a healthcare portal, outline critical transactions to monitor such as:

  • For a doctor- Opening a patient record, logging recent activity, updating prescription details
  • For a patient- Logging in, reading a doctor email, searching for care

All of these functions should be proactively monitored on a 24x7x365 basis (check out CitraTest APM!) to identify early problematic trends before your users are impacted.  After all, a cloud migration project introduces risks of performance degradation (e.g. noisy cloud neighbors in a multi-tenancy deployment resulting on “stolen CPU”, or processes that are not written to effectively use memory, which could overload the allocated memory in an Amazon EC2 instance).

With CitraTest APM you can continuously execute synthetic transactions, taking response time measurements along the way to quickly identify problematic trends. By automating the driving of ANY application — just like a real user – and visually examining the desktop and responding to changes, typing on a keyboard or on a mobile device or driving the mouse, CitraTest APM ensures that all critical aspects of an application are available and working effectively, alerting you in real-time at the first sign of trouble.
 
3. Decide polling frequencies and alerting policies. A good rule of thumb is to monitor key transactions more frequently (e.g. being able to send a sales proposal is more critical than reporting on sales pipeline) to identify performance degradation signs earlier, and define who should be alerted in the event of warning/critical threshold violations. With CitraTest APM, you can configure the number of response time violations that will trigger an alert to eliminate false positives, easily schedule monitoring tests or disable alerting on scheduled maintenance periods. You are in control of your monitoring!

4. Define custom reports Since different metrics are important for different stakeholders, take the time to map out role-based reports with custom information for each team (per application, per transaction, per functionality, etc.), and automatically distribute reports on an on-going basis (daily, weekly or monthly basis) to keep everybody informed and aligned.

5. Centralize IT response procedures and workflow. Don’t treat cloud-based application monitoring as a separate beast! After all, you are responsible for user experience for all applications hosted on your datacenter or in the cloud, so you want to add cloud monitoring to your existing IT monitoring framework to increase efficiency. Plus you want to use the same APM monitoring solution for all the applications you own and/or support.

Are you ready to boost performance for your cloud-based applications and find performance problems before your users do?

Check out CitraTest APM and CitraTest Mobile

Good luck with monitoring cloud-based applications!

Working on migration projects or DB rolls-outs? Simplify your life with automation!

Posted on by

Infrastructure roll-outs, cloud migration and new Websites top the list of this year’s IT projects. And let’s face it – even when everything goes according to plan (as if that’s ever the case!), these endeavors take up way too much of your precious time, keeping you from finishing other tasks, and keeping you for long hours in the office. The solution is simple – it’s time to streamline and AUTOMATE!

document_scanning

Think about automation as a series of steps that you can automatically generate and drive from a computer in sequence using a “virtual user” (copy, paste, type, click, generate a record, etc.), until a job is completed, WITHOUT any human intervention (other than building the automation script or a set of instructions that control virtual users behavior). For example, manual entry or database rolls-out are great examples of time-consuming, tedious and prone-to-errors tasks that you should automate to reduce costs, save time and accelerate project completion.

Let’s illustrate the point with Tevron’s real-world customer stories and ROI from the trenches:

Story 1:
Automate DB roll-outs (60,000 records in 4 days!
“I walked into a room and I saw 6 people in a room moving 60,000 records manually, countless weeks ahead them”. So I told them “Wait. I work in QA, I can build a CitraTest script and automate this task for you”.
The result? Instead of using a team of 6 working on this phase for months, the job was completed by CitraTest in only 4 days.
The pay-off? Smiles all around. I was able to save MANY days of work and effort, drastically cutting costs and accelerating time to completion

Story 2:
Automate Data Entry
“We were converting a legacy system into a new system and we found out too late that our vendor didn’t provide any tool to automate data migration. The team was desperate, many days, many man hours ahead of them to manually tackle this phase. CitraTest automatically completed the process and saved us a lot of time and efforts. Besides scripting anything (so you can automate all manual tasks), it has a nice recovery feature so you can re-start your data entry at any point without losing any data“.
The result? Peace of mind and complete accuracy, plus big savings in time and cost

If you are new to automation, speak with your QA/Testing counterparts to discuss how to go about building your automation scripts. For example CitraTest is a software test automation solution that can be used for both QA pre-production testing (functional, performance, regression testing, load testing) and to automate complex, tedious and time-consuming manual IT tasks. You can script against any application (Citrix, Terminal Emulation mode, legacy applications, all Web browsers, all Thick Client apps, etc.) that is accessible from a Windows client. CitraTest automates manual tasks by remaining a back-ground operation that generates real mouse activity and keyboard strokes and then responds to the images on the screen – just like a real user. Every user action can be replicated and automated.

Ready to learn more? Check out CitraTest!

The next time you walk into a room and see your peers engaging in manual tasks (or you are in the mist of completing tedious and time-consuming manual tasks), think about automation to get the job done, faster, error-free and much more cost-effectively. Great for both QA and IT teams.

Good luck!

Load Testing for IT Pros

Posted on by

If you think that load testing (or the act of stress testing your applications to ensure quality user experiences under heavy traffic) is only for your QA counterparts, think again. In 2013 a whopping 23% of IT budgets were allocated to application quality according to Capgemini’s World Quality 2013-2014 report, up from 18% in 2012. When you think about it, it makes sense. After all IT is the first line of response when users complain and application issues arise.
Blog #3

While load testing can be intimidating for IT pros at first, it shouldn’t be when you are armed with the right testing solution. Wouldn’t you prefer to triage and resolve overloaded Citrix/MTS issues early on, before large volumes of users are impacted? That is exactly what you can do with load testing, plus you can protect your organization of hefty compliance violation penalties and security breaches. For example in the summer of 2012, the University of Chicago Medical Center had an issue where overloaded Citrix servers were logging users under wrong profiles!

Here are some quick tips to help you get started:

1. Look for a non-intrusive performance testing solution to avoid performance impact in your production environments. Ideally, “load generating machines” should be used to generate user load against the server environment under test, while “measuring machines” should measure response times and gather server performance metrics on the back-end.

2. Select a tool that doesn’t require advanced programming skills. Unless you want to show off your programming skills, or move full time to development, look for a solution that offers an easy to use point-and-click scripting wizard, so you can easily build your test scripts.

3. Make sure that you will be able to replicate and measure REAL user activity using the ACTUAL client connection GUI for each load test. That way the most important metric is measured and analyzed – the end-to-end client side response time, so you can verify that SLAs are met for all pre-defined levels of load, plus is the most accurate way to identify underperforming application components at peak usage time. For example, each CitraTest VU test script executes on its own “desktop”, and opens its own client connection to the server-under-test, just as a group of real users would.

4. Double-check that you can create and execute large volumes of real user transactions against your production environment. That way you will be able to select the right level of load (or virtual users) to fit your custom needs. Since number of employees, locations, offices, peak requests, peak times, and so on will change over time, you need to have flexibility in your virtual user levels.

5. Build realistic and comprehensive testing scenarios. Consider areas such as key applications to load test, key multi-step user interactions to verify under load (e.g. logging, check a balance, add customer call log, etc.) , numbers of virtual users to test with, as well as ramp-up time. Don’t forget to generate realistic load modeling scenarios with varying number of virtual users for complete test coverage. In addition, as your real users will not all perform the same actions, you need to simulate multiple users concurrently performing different actions in order to realistically stress test your applications, and verify all critical functions and end-to-end transactions are working well under load.

6. Oversee both server performance and end-to-end response time under load. That way you can isolate hardware constraints and bottlenecks, and ensure sufficient back-end infrastructure capacity to handle all user requests and deliver quality user experiences at peak times.

7. Keep all stakeholders in the know. In a world where IT is becoming metrics-driven and ensuring alignment with your business counterparts is a must for success, you need to be armed with the right information. Share your load testing reports (test scenarios, results, virtual users, SLAs, etc.), and keep peers, management and stakeholders in the know.

8. Analyze results. Load testing gives you actionable insight to assess the need to tackle key projects for the business (e.g. capacity planning, new vendor selection, Web performance optimization, etc.). Get together with team members and stakeholders and periodically review load testing results to make better informed decisions.

9. Decide on-going load testing frequency to ensure IT readiness on a 24x7x365. Load Testing is not a “one-time-only” activity. In fact, you should think about load testing whenever you encounter any of these situations outlined below.
o Mounting user escalations (Increased number of open helpdesk issues, escalations and user complaints, intermittent application problems that you cannot reproduce or troubleshoot…)
o New application roll-outs (e.g. new application deployment, bug fixes, new patches or service releases)
o Organizational changes (e.g. mergers and acquisitions resulting in additional employees, offices, branches or locations, business development activities, geographical expansion)
o Infrastructure changes (e.g. datacenter consolidation, virtualization, new server deployments, cloud migration)

10. Check-out Tevron’s CitraTest VU to test Citrix/RDS performance under multiple load levels, and ensure quality experiences, for all users, at all times – under all load conditions.

Did you know that even Citrix relies on CitraTest VU to run their own load testing initiatives when a new version is released?

Your network is a living entity where the number of users, remote branches and locations, business requirements, load levels and users’ expectations change at lightning speed. So load test your applications (external facing and internal apps) to ensure IT readiness for whatever changes come your way! Need more convincing? Check-out how load testing helps you mitigate risks!

Good luck with your load testing efforts!

Stem the tide of user dissatisfaction with IT via application monitoring

Posted on by

You may not have noticed any changes in your organization yet, but it seems that dissatisfaction with IT is on the rise.  To give you some quick data points, 94% of IT projects are unsuccessful , a mere 15% of employees say that they are completely satisfied with their technology management department’s understanding of what they need to be successful, and only 13% of executives are satisfied that their IT organizations are very effective at introducing new technologies faster than competitors.

ID-100110223

As your role changes from tech gatekeeper to tech enabler, it is time to address the #1 customer complaint have about IT: “The application is not fast enough!”. Here are some quick tips to help you get the work done.

Tip #1: Monitor your applications from your end-user perspective.
Your applications are the lifeblood of your business, so they must consistency perform well, and perform fast. Otherwise, revenue and productivity will be lost, business operations will be disrupted, and user complains and escalations will go through the roof. An infrastructure-centric monitoring approach is not enough. Instead, you need to simulate and measure availability and response time of critical transactions on a 24x7x365 basis, from your user perspective, as shown below:

• User 1- CRM logging –> Open a customer record –>Add a sales log –>Log-out
• User 2- CRM logging –>Access sales pipeline –>Generate a report –> Log-out
• And so on

With proactive user-centric monitoring you can isolate problematic trends much earlier, and find and resolve issues before your users are impacted.

Tip #2: Proactively monitor all common workflows .
Take the time to map-out who your users are, and how they interact with your applications so you can monitor key user interactions. For example, with Epic EMR, both patients & doctors access MyChart records, so you should monitor Epic EMR response time from both patient and doctor perspectives, as shown below.

• Doctor– MyChart log-in –> Access a patient record–> Add visit log–> Save–> Logout
• Patient– MyChart log-in –> Access my record–> Search doctor phone information–> Logout

Also, if you are implementing or maintaining SAP, check out some additional SAP monitoring tips.

Tip #3: Find geographical response time discrepancies early on.
In some cases, employees in remote offices could experience slower response times than employees accessing your applications from headquarters. The faster way to find and troubleshoot problems like this is to monitor and compare availability and response time of your applications across multiple locations (Headquarters, Boston, NYC, remote office locations, etc.).

Tip #4: Validate and share SLAs reports with users and stakeholders.
In a new era where customer satisfaction rules, you need to continuously validate and demonstrate your SLAs. Therefore, you should track, report and periodically share your SLAs with management and stakeholders. With a metric-driven strategy in place you can keep stakeholders in the know, and quickly ascertain if you need to focus on app performance optimization or not.

Tip #5: Boost IT efficiency with a single solution to manage your entire application portfolio.
From legacy applications, to client server applications, to web applications, to home-grown custom applications, cloud applications or green screen apps, most large enterprises have a complex portfolio with 250-500 applications to support. The cost of purchasing, configuring and maintaining several monitoring products to support individual applications is too high. Plus lack of integration across monitoring consoles hinders troubleshooting and increases Mean-Time-To-Resolution (MTTR). Instead, look for one solution that lets you test and monitor all applications, regardless of where they are deployed.

Are you ready to boost user satisfaction with IT? Check out CitraTest APM!
CitraTest APM periodically executes synthetic transactions, taking response time measurements along the way. By automating the driving of ANY application — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse– CitraTest APM ensures that all critical aspects of an application or service are available and working effectively, alerting you in real-time at the first sign of trouble – before your users complain.

Good luck!

SAP Monitoring DO’s and DONT’s

Posted on by

As an IT pro supporting SAP, or tackling an SAP implementation, simplicity is certainly not the first attribute that comes to mind. In fact, as an enterprise suite of ERP, CRM, PLM, SCM and SRM applications, SAP links and integrates every business process across your organization. While 253,500 customers in 188 countries successfully rely on SAP to accelerate their business operations, inadequate planning, lack of testing during each implementation phase, improper SAP monitoring, or even poorly managed user training can all result in costly pitfalls and mistakes for you and your organization.

For example, a recent $50 million SAP implementation by San Diego — the biggest IT project ever undertaken by the city, designed to provide a core IT architecture for 15 years — ended up tripling employees’ workloads for certain types of tasks! Pair this up with other highly visible SAP implementation failures at Marin County, or at Hershey’s where order fulfillment time doubled to 12 days and you will get the point. Testing and validating SAP functionality and workflows during each implementation phase, and proactively monitoring availability and response time of SAP in production systems, as perceived by your end-users, are critical to your success. Here are some tips to help you get started.

PRE-IMPLEMENTATION GUIDELINES.

• DO #1. Document existing business processes: Since SAP touches so many business functions, take the time to understand and document how each business user (project managers, HR, sales, financial teams, etc.) completes existing processes and common workflows.
SAP Workflow

• DO #2. Redesign business processes. Many of the SAP implementation pitfalls we just mentioned were primarily due to simply replicating existing processes instead of redesigning them with SAP for more efficiency. Therefore, don’t skip this crucial, yet time-consuming step, and map and optimize all user requirements, for each group of users, into new workflow models. This information will also help you guide your monitoring efforts later on

• DO #3: Reduce the number of customized functions. Try to use as many objects and functions as you can from the off-the-shelf version of SAP instead of writing your own custom code. You might run into quality issues with custom objects later on, plus they might not work as expected if/when you roll-out a new version of SAP.

• DO #4: Test and validate each SAP implementation phase – from an end-user perspective. Begin manual testing as early as possible, so users can provide feedback on SAP business processes and validate reporting structures. Later on, automate your testing processes to save time, increase efficiency and thoroughly test and re-test as needed to ensure quality and top-notch SAP performance. Look for a testing tool where you can easily replicate and automate end-to-end user interactions with SAP in the form of testing scripts (e.g. enter an order, generate a shipping receipt, create an invoice) , which can be later on re-used for SAP production monitoring as well.

• DO #5: Train your users. Tailor your training accordingly, and plan for role-based training customized to each set of users’ unique requirements

SAP PRODUCTION MONITORING GUIDELINES.

Did you know that the #1 customer complaint faced by organizations that use SAP is “SAP is too slow”? Therefore, instead of infrastructure-centric SAP monitoring, look for synthetic monitoring capabilities to automate the driving of your SAP applications — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse and continuously taking response time measurements along the way, on a 24x7x365 basis, so you get alerted at the first sign of trouble, before your real user are impacted. Here are some additional tips for you

• DO #6. Build a comprehensive Monitoring Plan. Just like you documented your business processes and workflow for each set of business users, take the time to plan and document your SAP monitoring strategy, including key areas like: Monitoring scripts (so you can proactively measure complete end-to-end user interactions with SAP), Alerting Policies, Notification Groups, Reporting Distribution, Monitoring Locations, Service Level Agreements and so on

• DO #7. Focus on Essentials workflows first. Tackle the basics, including key financials, HR/payroll, supply chain processes and workflows, and add more monitoring scripts later. In addition, consider using a dedicated “SAP user” monitoring account to keep your SAP data secure.

• DO #8. Look for non-invasive agentless SAP monitoring that will not change your application code. That way you can ensure no impact on your SAP production system, and complete monitoring accuracy

• DO #9. Be aggressive in your SAP monitoring thresholds (availability and response time). Since proactive SAP monitoring helps you identify problematic trends early on, before your end-users are impacted, you should baseline and setup both warning and critical thresholds for all monitored SAP transactions. Consider as well monitoring SAP transactions across multiple locations so you can easily identify response time discrepancies across users and locations.

• DO #10. Don’t monitor SAP in isolation. Integrate SAP monitoring, alerting and performance dashboard views within your existing IT management tools and frameworks tools to increase IT efficiency and streamline your response procedures

Are you ready to avoid SAP problems and ensure success?

Check out CitraTest (SAP test automation) and CitraTest APM (proactive SAP response time monitoring from a user perspective).

Good luck with your SAP efforts!