Home

The Value in Values

James Thomas' blog - Wed, 16/01/2019 - 05:53



The testers at Linguamatics decided to explore the adoption of a set of team values and this short series of posts describes how we got to them through extended and open discussion.

If you find the posts read like one of those "what I did in the holidays" essays you used to be forced to write at school then I'll have achieved my aim. I don't have a recipe to be followed here, only the story of what we did, in the order we did it, with a little commentary and hindsight.
  • Introduction
  • Why?
  • Teasing Them Out
  • Living With Them
  • Reflection
--00--
Our team provides testing services to other teams in the company, in their contexts. That means we cover a selection of products, domains, and technologies across several development groups, operations, professional services projects, our internal compliance process, and more.

In terms of methodology, we are in permanent Scrum teams, we join time-bounded projects set up to implement a particular feature or satisfy a particular customer need, and we work day-to-day with groups whose priorities and resources are subject to change at very short notice.

In spite of the varied nature of our assignments it's historically been our desire to maintain strong team bonds and an information-sharing culture and so we've engineered some formal and informal opportunities to do that.

Amongst other things, each week we have a catch-up with some kind of presentation (such as a feature, a tool, an approach), we have a daily stand up (roughly: prefer outcomes over outputs, share issues, ask for data or help), and we have a tradition of optional, opportunistic, 5-10 minute overviews on topics that are potentially interesting right now but too deep for stand up.

We also have a regular team retrospective in which we allow ourselves to discuss pretty much anything about the way we work. It tends to stay out of project specifics — because they'll be discussed within the projects — but recent topics have included dedicating time to shortening the run time of a particular test suite to enable developers to get faster feedback from it, creating a specific type of virtual machine for us to share, and reviewing how we schedule work.

At the start of 2018, a retro topic that I proposed after hearing Keith Klain speak at Quality Jam 2017 was voted up. In his talk, Keith said that one of things he likes to see in a team is a shared understanding of the important factors that frame how they work. Based on that, I asked should we establish a set of team values, principles, or a mission statement?

The resulting discussion generated enthusiasm. And questions, naturally. They included:
  • What do we want to create: a defined mission? principles? values?
  • ... and how do these things relate to one another?
  • It shouldn't be be too low-level; it should apply across teams, projects, and so on.
  • It shouldn't be restrictive or prescriptive; there should be flexibility.
  • It should be a framework for decision-making, not a decision-maker.
  • Do we really need anything different to the company values?
  • Do we want it to change the way we work, or encapsulate the way we work?
  • Do we want others in the company to see it?
  • ... and might it change how others see us?

None of us had ever tried to externalise group values before so we began by researching what others had done. Here's a few examples from within the testing space:

Some of these were published after we started so didn't have as much chance to influence what we did. Iain McCowatt's Principles Not Rules was inspiring to me, but is unavailable as I write this. It's such strong material that I've left the links in the list above in the hope that it'll come back. Small comfort: I saw his talk on the same topic at EuroSTAR 2015 and a handful of my notes are here.
Outside of testing, in development and more generally, we looked at pieces like these:

Closer to home, we observed that our company has some useful data to contribute: our corporate values published on the internal wiki, and a set of informal values that are regularly called out verbally at all-hands meetings.

Finally, we looked to see whether values are encoded implicitly in our tester job adverts, which include lines like these:
  • We strive to provide relevant information to stakeholders and we're flexible about how we do it.
  • We use and we eagerly solicit peer review, we’re open to new ideas, and we perform regular retrospectives to help us improve ourselves and our work.
  • Our company respects what we do, and we’re a core part of our company’s work and culture.
  • Linguamatics is active in the local testing community, regularly hosting meetups and Lean Coffee.
  • We have regular in-house training in testing and other skills.
  • If you get the job you will be expected to
  • ... take responsibility for your work,
  • ... apply intelligence and judgement at all times,
  • ... be able to justify your position and be prepared to discuss alternatives,
  • ... look for ways to improve yourself, your work, the team and the company.

To summarise how we started down this road, then:
  • We wondered if we should think about making our implicit shared values explicit.
  • We discussed it, and decided that we'd give it a go.
  • We did some research to see what was out there, and what we already had.

In the next few posts I'll describe how we moved from this point to a set of values that we can agree on as a team.
Image: https://flic.kr/p/oGMUQ
Categories: Software Testing

Breaking the Test Case Addiction (Part 1)

Michael Bolton - Wed, 16/01/2019 - 01:11
Recently, during a coaching session, a tester was wrestling with something that was a mystery to her. She asked: Why do some tech leaders (for example, CTOs, development managers, test managers, and test leads) jump straight to test cases when they want to provide traceability, share testing efforts with stakeholders, and share feature knowledge with […]
Categories: Software Testing

A Tactical Automation Case Study - Java, JavaScript, API and Dev Tools

Alan Richardson's Blog - Mon, 14/01/2019 - 07:00

TLDR; Automating is not limited to “Test Automation”. With the ability to code, we gain the flexibility to approach problems in multiple ways. We can refactor tactical code to become strategic code.

I needed to migrate some data from one system to another. To do it manually would have taken me about 1.5 to 2 days. I automated tactically and was done in 2 hours.

I have a full video report of this work on Patreon

I needed to automate migration of URLs from one system to another.

Since I was only ever going to do this once, I consider this tactical automating.

Getting Data Into A System

The System I needed to move data to did not have:

  • a bulk import function
  • a formal API

But, it is a web application, and it has a GUI.

This means that if I can model what the GUI does then I might be able to automate it. And I don’t mean automate the GUI, I mean automate the messages that the GUI sends.

In Automating And Testing a REST API I referred to this as “App As API” and the book contains a longer write up of this approach.

Basically:

  • I use the GUI
  • I have all web traffic going through an HTTP Proxy (I used BurpSuite)
  • I determine what are the minimum necessary HTTP messages to replicate
  • I copy and paste those messages into an IDE
  • I automate those messages directly
  • I refactor them to have the paramaterised data that I want to upload
  • I create the data in an Array
  • I loop over the array and send the messages
  • I send all messages through a proxy so I can see the outcome and the content
  • Initially I run everything in debug mode so that I can check if it is working and review the outcome in the System GUI.

Note:

  • The App doesn’t have an API, I sent through the HTTP messages that the GUI sends
  • I don’t attempt to replicate all the messages, just the ones that make a functional difference
    • with many apps you’ll find logging calls, diagnostic calls, health check calls, etc.
    • I just used the ones that trigger functionality

The code is simple:

  • I have hardcoded strings
  • I refactor those into parameterised Strings that use a String.format
  • I did not create any payload objects because I’m working tactically rather than strategically
    • If I was automating something complicated then I might use payload objects, but I didn’t need to.
  • I needed an object to represent migrated data
    • I used a private inner Class
    • I often do this for initial work, then refactor to a public Class when other test classes need to use it
  • I created a data class, a POJO with a constructor, where all the fields are public final and set by the constructor
    • because I didn’t need the overhead or protection of methods for the fields
    • I only needed a single constructor
    • e.g.
    class ShortUrl{
        public final String path;
        public final String url;
        public ShortUrl(String path, String url){
            this.path = path;
            this.url = url;
        }
    }
  • I sometimes start my strategic automated code with inner classes and then refactor to public classes, and refactor the public fields to private fields with public accessor and setter methods.
Getting Data Out of a System

I tried to use the “csv export” from the System, but it didn’t have all the fields I needed.

I raised this with the vendor, and they confirmed it was a bug that they will fix, but I’ve already worked around the issue. Tactical automating is often used to workaround bugs and issues.

This is a JavaScript heavy GUI, so I can’t use an HTTP request to parse the data from the HTML.

In these circumstances I automate in the Dev Tools Console.

Useful JavaScript constructs to learn:

Also a grasp of CSS Selectors can help.

For Parsing data I do like Regex:

let regexp = /.*\/(.*)\/(.*)/ig;    
let matchOne = regexp.exec(notes);
var project = matchOne[1];

Very often when I’m working with JavaScript from the console I end up building a string as I loop over lots of data:

var reportLine = '"'+url+'" => "'+shortUrl.trim()+'",';
console.log(reportLine);
output = output + reportLine +"\n";

I often console.log it so I can see it for debugging, but by building a string output it is easy to output it to the console and then copy and paste into another System. If I console.log every line then copy and pasting the output is a pain.

Working tactically means:

  • I write code that works
  • it may not do everything at once, so I have to change variables, tweak conditions etc. when I want to run against different data sets
  • I will probably run it multiple times
  • it is supporting what I am doing rather than trying to automate a workflow reliably from start to finish
More Regex

I also used Regex find and replace in Visual Studio Code. This is becoming my favourite Markdown and text editor.

I find this useful as a tactical way to convert code from one language to another

find: (.*)=> (.*), replace: new ShortUrl($1,$2),

The above allowed me to convert a PHP Array to a Java Array of objects.

Mix of Skills

To automate like this I have to use:

  • Technical Web and HTTP skills and tools that I have built up from testing
  • Java and Libraries that I use for automating strategically
  • JavaScript which I frequently use for in browser automating and adhoc GUI querying and manipulation

All of the code was tactical.

  • I haven’t saved it
  • It wasn’t intended to be maintained
  • I refactored it only as far as I needed to make my work easier
  • It wasn’t intended to run reliably, sometimes I have no synchronisation and execute it in debug mode to have synchronisation points
  • It has minimal error handling and checking, it relies on me spotting problems to stop execution

Tactical is often a first step to creating strategic automated execution, and with more refactoring I could have created a long term strategic automated migration. But I didn’t need to.

No doubt some of the code snippets I’ve included here seem like ‘bad practices’. And they might be if this was strategic.

If you want to see this in action then I have a video in Patreon, but hopefully there is enough here to encourage you to learn some of the tactics and approaches that I used for tactical automating.

Categories: Agile, Software Testing

Software Performance Predictions for 2019

Test And Verification Services Blog - Mon, 14/01/2019 - 05:16

The good news for software performance is that there are many emerging trends that will take software performance to the next level. This article elaborates the major trends which are expected to shape the software development industry in 2019.

Read More

Find out how T&VS Performance Testing services help you to solve issues in a cost-effective way and deliver with high quality.

The post Software Performance Predictions for 2019 appeared first on T&VS.

Categories: Software Testing

Why Is Application Security Testing Orchestration (ASTO) A Need?

Test And Verification Services Blog - Mon, 14/01/2019 - 05:13

Most organizations are using both SAST and DAST tools together to increase confidence in the results of their scanners.ASTO offers a single platform for both parties where they can see outcome of the scans run on projects. This article elaborates how ASTO can be used to improve application security.

Read More

Find how T&VS Application Security Services enable you to prevent breaches by protecting and securing your data against web attacks, and vulnerabilities.

The post Why Is Application Security Testing Orchestration (ASTO) A Need? appeared first on T&VS.

Categories: Software Testing

Why the Smart Home needs to get even Smarter

Test And Verification Services Blog - Mon, 14/01/2019 - 05:11

Consumers want smart home devices that just reliably work regardless of manufacturer or vendor ecosystem.The smart home is a wonderful, expanding, highly inventive, new industry sectorbut right now, it’s a mess. This article elaborates why the smart home devices needs to be smarter.

Read More

Learn more about how T&VS IoT services and solutions can ensure the smart home revolution is delivering safe, secure and compliant products.

The post Why the Smart Home needs to get even Smarter appeared first on T&VS.

Categories: Software Testing

Making IP Friendlier

Test And Verification Services Blog - Fri, 11/01/2019 - 05:52

Most standard protocol and interface IP enable verification engineers to check basic features, such as system start-up. VIP enables more detailed exploration. This is becoming increasingly important because of the growth in complexity of system-on-chip (SoC) designs. This article shows how to make IP more user friendly.

Read More

Find out how T&VS have developed a unique process that enables companies to make continuous improvements to their design and verification environments.

The post Making IP Friendlier appeared first on T&VS.

Categories: Software Testing

Pros And Cons Of Agile Testing

Test And Verification Services Blog - Fri, 11/01/2019 - 05:49

Agile testing refers to making changes to the existing project during its development lifecycle as per the project requirements.It is an incessant process which is done during the different stages of the project. This article outlines the benefits and risks involved in agile testing.

Read More

Find out how T&VS Software Testing services help you to establish a cost-effective software testing facility that delivers improved quality, reduces risks and time-to market.

The post Pros And Cons Of Agile Testing appeared first on T&VS.

Categories: Software Testing

Success is in the Scoping: Incorporating Penetration Testing into Cybersecurity Planning

Test And Verification Services Blog - Fri, 11/01/2019 - 05:47

Penetration tests can fail before they even start, thanks to poor scoping. It’s easy to hamstring pen testers through overly restrictive scoping, but a scope that’s too wide can end up reporting back what is already known. This article outlines how to incorporate penetration testing into cybersecurity planning.

Read More

Understand how T&VS Penetration Testing services help you protect & defend against latest and future attacks and maintain compliance, eliminate IT security threats, & can reveal how hackers may breach systems.

The post Success is in the Scoping: Incorporating Penetration Testing into Cybersecurity Planning appeared first on T&VS.

Categories: Software Testing

How IoT Is Shaping The Smart City

Test And Verification Services Blog - Fri, 11/01/2019 - 05:45

Smart cities will leverage advanced connectivity to serve their citizens and they will customize their solutions to their needs on a person-to-person basis. This article outlines how IoT technology is helping to build future smart cities.

Read More

Find out how T&VS innovative solutions are helping cities to become smarter.

The post How IoT Is Shaping The Smart City appeared first on T&VS.

Categories: Software Testing

A Moment of Jerry Weinberg Zen

Michael Bolton - Thu, 10/01/2019 - 18:21
The year was 2006. James Bach and I were running a workshop at the Amplifying Your Effectiveness conference (AYE). We were in one of those large-ish, high-ceiling conference rooms with about 15 programmers and software consultants. We were showing them one of James Lyndsay’s wonderful testing machines. (You can find it here, but you’ll need […]
Categories: Software Testing

Chip Design Verification: It’s All About the Coverage

Test And Verification Services Blog - Thu, 10/01/2019 - 07:24

In the gate-level design era, some chip development teams looked at stuck-at-fault metrics to gauge how well their tests were exercising the design. However, fault simulation is slow and expensive and comes too late in the project. This article explains how chip design verification helps to avoid missed design bugs.

Read More

Find out how T&VS Verification services help to meet the challenging requirements with respect to performance, flexibility and verify today’s complex designs effectively.

The post Chip Design Verification: It’s All About the Coverage appeared first on T&VS.

Categories: Software Testing

6 Steps To Eradicate Mistakes Using Mobile Test Automation Strategy

Test And Verification Services Blog - Thu, 10/01/2019 - 07:22

Mobile testing is a very critical area for businesses. Therefore, teams are working to make it effective by incorporating automation strategies. This article shows how to avoid mistakes using mobile test automation strategy.

Read More

Find how T&VS Test Automation Services enables organisations to increase the effectiveness, efficiency, and coverage of their software testing.

The post 6 Steps To Eradicate Mistakes Using Mobile Test Automation Strategy appeared first on T&VS.

Categories: Software Testing

Moving towards achieving Autonomous Vehicle’s Functional Safety

Test And Verification Services Blog - Thu, 10/01/2019 - 07:20

Electronic systems are becoming more common and sophisticated. Because of this, safety and verification are a critical part of every automotive chip design as well as the IP integrated into the silicon. This article shows how to improve and achieve autonomous vehicle’s functional safety.

Read More

Learn more about how T&VS Automotive Verification and Test solutions help to address the challenges of delivering safe, secure and compliant automotive products.

The post Moving towards achieving Autonomous Vehicle’s Functional Safety appeared first on T&VS.

Categories: Software Testing

What Is IoMT and How Does it Work?

Test And Verification Services Blog - Thu, 10/01/2019 - 07:18

When the data collected with IoMT devices are combined with electronic health record (EHR) systems, a new dimension is opened and many possibilities and uses are born in which this technology can play a pivotal role in transforming modern healthcare systems, making them more efficient and robust. This article shows how Internet of Medical Things work.

Read More

Learn more about how T&VSIoT driven connected healthcare services can help organizations innovate and improve patient satisfaction and boost treatment outcomes.

The post What Is IoMT and How Does it Work? appeared first on T&VS.

Categories: Software Testing

T&VS to present Formal Verification Bootcamp at DVCon USA on Feb 25, 2019

Test And Verification Services Blog - Wed, 09/01/2019 - 15:59

During the last week of February, the DoubleTree hotel in San Jose, California will once again host DVCon U.S.

T&VS are delighted to be contributing to the program by presenting a ‘Formal Verification Bootcamp‘ where delegates can enhance their knowledge of formal verification with hands-on examples using SystemVerilog assertions.

Formal Verification Bootcamp
  • Speaker: Michael Bartley, CEO and Founder, Test and Verification Solutions
  • When: February 25, 1:45pm – 3:15pm
  • Where: OAK Suite, DVCON USA, San Jose, CA.
  • Format: Short Workshop, including worked examples

The Formal Verification Bootcamp is for design and verification engineers looking to enhance their knowledge of formal verification and to learn how to write effective assertions in SystemVerilog to find and fix bugs. The workshop explains how formal verification differs from simulation, the advantages and drawbacks, and how to effectively incorporate formal into a design flow. The workshop uses a small worked example and the delegates prove some ready-made SystemVerilog Assertions using a tool during the workshop. Delegates are also asked to extend the assertions and to also use cover properties to visualise the design and to measure coverage. The workshop ends with a thorough analysis of the strengths and weaknesses of formal verification. It shows how others have exploited formal and successfully incorporated it into their design flow.

Delegates should leave with enough knowledge to be able to understand how they could potentially adopt formal verification and perform an effective evaluation of tools.

Read More at DVCON Arrange a Meeting

If you would like to arrange to meet-up with the T&VS team at DVCon for an informal chat about your next verification project please contact us to arrange a suitable date and time, or simply call one of our local offices.

Arrange a Meeting DVCon at-a-Glance

The post T&VS to present Formal Verification Bootcamp at DVCon USA on Feb 25, 2019 appeared first on T&VS.

Categories: Software Testing

Not Get Fired for Choosing RISC-V

Test And Verification Services Blog - Wed, 09/01/2019 - 08:29

One of the big benefits of RISC-V is that the architecture is open source. The RISC-V architecture has limited targets while the ecosystem is being developed. This article outlines how to adopt RISC-V without risks and a briefing of RISC-V Summit 2018.

Read More

To help you deliver successful RISC-V based designs T&VS offer specific services that build on and extend the world-class test and verification services that we have been delivering to the semiconductor industry.

The post Not Get Fired for Choosing RISC-V appeared first on T&VS.

Categories: Software Testing

How “Shift-Left” Testing Can Help Your Product Quality

Test And Verification Services Blog - Wed, 09/01/2019 - 08:26

There were a significant number of risks that were involved by keeping testing as a one-off activity since it had a lot of implications on the project deadlines, as well as the cost. This realization gave rise to the concept of Shift Lift, shifting the testing phase to the left. This article explains how Shift Left testing can help to improve the quality of the product.

Read More

Find out how T&VS Software Testing services help you to establish a cost-effective software testing facility that delivers improved quality, reduces risks and time-to market.

The post How “Shift-Left” Testing Can Help Your Product Quality appeared first on T&VS.

Categories: Software Testing

IoT in Healthcare: Managing The Opportunity & The Risk

Test And Verification Services Blog - Wed, 09/01/2019 - 08:23

IoT in healthcare can increase hospitals’ efficiency in many ways. But should also be prepared for the risks it brings. This article explains how to manage opportunity and risks involved in healthcare industry caused by IoT technology.

Read More

Learn more about how T&VSIoT driven connected healthcare services can help organizations innovate and improve patient satisfaction and boost treatment outcomes.

The post IoT in Healthcare: Managing The Opportunity & The Risk appeared first on T&VS.

Categories: Software Testing

Industrial IoT – Reducing Operation Costs and Minimizing Human Error

Test And Verification Services Blog - Wed, 09/01/2019 - 08:20

Minimizing the damage done by human errors must be a top priority for businesses worldwide. Industrial IoT can greatly mitigate the problems caused by human error and help businesses improve processes and save money. This article explains how industrial IoT helps to reduce operation cost and minimize human error.

Read More

Find out how T&VS IoT services help to build a foundation of trust, security and safety in theindustrial IoT devices.

The post Industrial IoT – Reducing Operation Costs and Minimizing Human Error appeared first on T&VS.

Categories: Software Testing

Pages