A State-of-the-Art Virtual Inspection Methodology Using Mixed Reality Technology Gets High Marks from European Medicines Agency Inspector

This is Part 2 of our four-part series on virtual inspections we review a state-of-the-art virtual inspection platform using mixed reality technology.

In Part 1, former FDA executive Howard Sklamberg reviewed the current inspection landscape and what may trigger an inspection these days.

In Part 3 of our series on virtual agency inspections, former FDA investigator Ricki Chase will provide her insight into the best way to prepare for virtual inspections, including how to always be ready, how to go about reviewing documents, and which documents FDA is likely to request for review. In Part 4, Chase will provide advice on how to host a virtual inspection.

Part 2 of the series focuses on Thermo Fisher Scientific and its use of virtual reality and mixed reality platforms to create a state-of-the-art system that it has successfully used in virtual inspections, including with both clients and the European Medicines Agency (EMA).

At the ISPE Biopharmaceutical Manufacturing conference held virtually in early June, Thermo Fisher Scientific Regulatory Affairs Manager Monica Commerford discussed how her firm built and uses the system in a presentation titled, Virtual Inspections: Navigating the New Paradigm.

“We have already been using this technology for our clients,” she explained. “And we recently concluded an EMA inspection at our Alachua, Florida site. Our experience has shown us that this is a viable platform that will really revolutionize how we perform inspections.”

Commerford noted that the technology itself “has not gone unnoticed by the regulatory authorities. In fact, the EMA inspector said, ‘This has been a new thing for us and has been a positive experience…you may not realize it, but I have this on a bigger screen, and the audit is almost life-size!’ We could not find a better endorsement.”

In her presentation, Commerford provided an overview of the current regulatory landscape and why virtual inspections are a useful resource, the advantages that virtual inspections provide over traditional onsite inspections, the technology used, and what she thinks regarding the use of virtual facility inspections in the coming years.

[RELATED: How does your inspection rate compare to that of your peers? If you want to find out, get your Free Benchmark Report.]

Using Virtual Inspections to Maintain Commitments

The Thermo Fisher manager maintained that one major challenge during the COVID-19 pandemic has been how to allow regulatory authorities to keep their commitments to the public by performing necessary inspections.

“Both the FDA and the EMA have scaled back regulatory inspections. In its statement to the public, FDA indicated that it may evaluate records in lieu of an onsite inspection. Also, the EMA has been considering allowing remote assessments of compliance.”

Pharmaceutical companies, Commerford said, have reached out to FDA and suggested that virtual inspections may be key and the way to go to allow for CGMP verification. “Virtual inspections during a pandemic can really help to ensure continuous supply of much-needed drugs, not just for the United States, but worldwide.”

She explained that the virtual inspection system her company developed “has allowed us to maintain business continuity without sacrificing client and regulatory responsibilities.”

What Technology is Used?

The heart of the virtual inspection system, Commerford explained, includes a 360 degree camera mounted on an IV pole with wheels, the Microsoft HoloLens virtual reality device, Avatour software, and the use of Microsoft Teams software to review documents during inspection.

“For our setup, we would simply attach the camera to a cell phone or iPad and mount it on an IV pole. This pole can be wheeled around a facility on smooth surfaces allowing real time online video streaming tours of the GMP facilities. And a camera itself has a high enough pixel resolution to read documents and labels and logbooks during the tour. The other advantage to using this type of setup is its cleanability. If needed, you can go in and out of rooms with this if you have the appropriate cleaning studies performed.”

With this technology virtual inspections can occur in all GMP spaces. However, depending on the floor surface, “you may need to transition to another piece of technology. So, for example, you do not really want to roll an IV stand around on a bumpy surface. Or there may be some surfaces or activities that are not accessible by this 360 degree camera.”

“In these cases, we can use the Microsoft HoloLens,” Commerford explained. “This is a piece of virtual reality technology that allows us to show and perform activities that we may not be able to show otherwise using the 360 degree camera. For example, showing an activity in the biosafety cabinet or opening filters to show how clean they are is something that the HoloLens can be used for.”

Figure 1

In addition, as part of the virtual inspection experience, one other piece of technology that is used is the Matterport 3D technology mapping. This is commonly used by realtors to virtually map homes for online viewing. Thermo Fisher Scientific has virtually mapped its facilities to orient auditors and inspectors to the facility prior to giving them the virtual livestream tour using the 360 degree camera.

Figure 2

The Matterport 3D technology is only used to orient visitors to the facility and should not replace the livestream tour for the inspection, Commerford maintained. “The live use and streaming using the HoloLens or the 360 degree camera with the Avatour software is really where the action is.”

She displayed a video of what the Matterport 3D mapping looks like and commented that the resolution is excellent. (For an example of how this technology works, see 3D Mapping the Exploratorium with Matterport)

According to Avatour’s website, “The Avatour platform allows remote guests to visit a real place, in real time, using inexpensive mobile 360° capture technology. You will need a low-cost 360° camera, along with a selfie stick and audio headset. These tools enable you to give live virtual tours while speaking directly with your remote guests.”

“Using the Avatour Host app on your smartphone or tablet, schedule a time for clients and participants to meet. They receive calendar invites. Guests can attend using any modern web browser, or with a VR headset for a fully immersive experience. At the meeting time, up to 5 guests from anywhere in the world can join you virtually, using either a VR headset or any mobile device or desktop.” For an example of what an Avatour experience looks like, click here.

Commerford pointed out that the combination of the 360 degree camera using the Avatour software that can be live streamed using the HoloLens and 3-D mapping along with the document review using Microsoft teams allows for virtual inspections that rival in-person inspections.

She pointed to the importance of the document review piece, in which documents are organized in a way that facilitates review by inspectors and allows document storage in a controlled manner. “While Microsoft Teams can be viewed on a native application, as in software on your computer, it can also be viewed solely for the web. We found this to be an advantage during our recent regulatory inspection,” she explained.

Using this process, the company had control over the documents, and their recent experience was that EMA “was accepting of viewing files that were temporarily on the Web,” Commerford said.

She explained that the “web-based browser has the same functionality as native software on the computer, but that the risk of downloading viruses is low and having other issues is lower because both regulators and organizations have their own tools in place to ensure that their own internal IT infrastructure is not compromised.”

Implementing Virtual Inspection Strategies and Lessons Learned

During the implementation and conduct of virtual inspections, Thermo Fisher discovered similarities and differences in roles and responsibilities, security concerns, and digital support needs as compared to traditional inspections. Instructive lessons were also learned and shared.

Commerford said the same roles typically apply between onsite inspections versus virtual inspections. For example, there will still be escorts, scribes, a ready room, and a strategy room. In addition, a new player becomes important – digital support. “This group becomes key now to help ensure a smooth experience for ourselves and for either client auditors or inspectors.”

As in a traditional in-person inspection, who has access to what areas and data is important. In a traditional audit, a person can be physically posted at a door to control access to the strategy room or ready room. Here, controlled access must be performed digitally.

To ensure a smooth inspection experience, Commerford recommends performing wireless access datapoint mapping to gauge the wireless signal strength in all the manufacturing areas. “Our site already had a long history of using cell phone communication between rooms. So, the concern for signal strength was low for us,” she explained.

Also recommended is having multiple 360 degree cameras that can run Avatour software staged throughout the site at places you know you will be touring and that inspectors will want to view. This will reduce cleaning and transition time and minimize dead airtime during the tour.

The use of 5G LTE, Commerford said, “lends credibility to your inspection. For example, when the power goes out, the inspectors are going to able to see what you do when the power goes out and what activities go on. Also, logistically, you do have to think about evaluating the ability of your equipment if you are using the types of equipment that we are we are using.”

Lastly, she recommended to emphasize to all participants, whether they are inspectors or client auditors, that a live inspection and a live tour are being conducted and the same rules apply regarding recordings or screenshots as during an in-person inspection.

Pre-meetings and Training Also Facilitate a Good Inspection Experience

Other lessons learned when conducting virtual inspections include the importance of training participants in the virtual reality and mixed reality platforms, ensuring participants have the appropriate computer skills to participate in virtual document reviews, and of pre-meetings to convey expectations and areas of concern or interest.

Figure 3

Commerford emphasized the importance of training on the use of the technology being employed, and how mock inspections help with learning the tools and knowing what to inspect. For example, if staff will be using the HoloLens to perform an activity or to show a space that is not accessible by the camera, it is important to mock that ahead of time to help the team get comfortable in the virtual reality environment.

She pointed out that inspectors and auditors may also need to be trained to a certain degree. It is important that they have the computer skills needed, for example, to perform a virtual document review. This is one item that can be discussed in a pre-meeting.

Pre-meetings with clients or auditors also help the company gauge their level of understanding of the software – for example, how much help they may need to navigate the platform. Pre-meetings help build credibility for the platform, “so they can see that it is just like an onsite inspection and they are able to get access to the information they need,” Commerford said.

“Pre-meetings also provide you with opportunities to learn ahead of time without the pressure of being in an inspectional environment, to start building that rapport that you sometimes lack when you have virtual meetings, whether it is through Zoom or other types of software,” she said.

In the pre-meeting it is important to talk about pre-inspection document requests and to schedule what the inspectors want to see and when during their tours or during the inspection. This level of planning up front helps ensure that the inspection runs on time.

Advantages and Disadvantages

While there are some advantages and disadvantages to both in-person onsite and virtual inspections, Commerford maintained that virtual inspections are superior in many ways.

Figure 4

Both take up a similar amount of time, the pre-meeting notwithstanding, she said. “And in our recent EMA inspection, we had four half-days rather than two full days of inspection. This allowed us to maintain work during business hours and to use the other four hours to answer questions or to prepare.”

In either type of inspection, the final day is typically a wrap up to share findings and help ensure closure of any open items.

“In terms of time, we have also found that the inspectors like to stick together and cover similar material, which did lead to a slightly slower inspection dynamic, whereas in a traditional onsite inspection they may use a divide and conquer type approach,” Commerford said.

Performing inspections virtually also allows holding the inspection over nonconsecutive days, which helps to ease the burden of the hosting responsibilities.

Commerford noted one different inspection dynamic: Questions that were typically posed during the normal discourse of the tours or during sit downs or in the audit rooms or the inspection rooms were frequently replaced by quick questions that become part of a daily list that inspectors provide to the sites at the end of the inspection day.

“You may see this as a disadvantage because that may slow the inspection down a little bit,” she said. “But we found this to be the advantage for us because we had time to prepare responses and present those responses in our next interaction or the next inspection day. And if it was a nonconsecutive inspection day, that gave us more time to prepare.”

Virtual inspections during a pandemic can really help to ensure continuous supply of much-needed drugs, not just for the United States, but worldwide.

Another advantage is that subject matter experts (SMEs) from multiple sites can participate in a virtual inspection and be able to answer questions live without traveling.

One disadvantage is the inability to read body language. “Sometimes it is hard to get a great read on the status of the inspection. So, we asked questions regarding whether we could move on.”

As previously mentioned, there is some prep work associated with the training to use the software and the technology. “But this initial work is needed for the experience,” Commerford pointed out. “And once you put the work in and it becomes standard, it simply just needs to be maintained rather than repeated.”

It is also important to have the use of the equipment reviewed by company health and safety personnel to ensure that the operators are trained appropriately in the safe use of the equipment. “When you are in a virtual reality environment, you may experience some instability while walking, for example,” she said. “So, your safety team really needs to be involved and you may want to follow their guidance on using augmented reality in difficult or dangerous situations.”

Commerford maintained that a lot of the disadvantages are because the use this platform is in its infancy, and that as it is used more the disadvantages will be overcome.

“What success looks like to us is that the auditors can adequately assess the environment and have access to effective procedures and SMEs,” she said.

“During the close out for our recent EMA inspection, the EMA inspector even used the phrase, ‘during the tour.’ It was not qualified with ‘virtual’ or ‘live stream,’ simply ‘tour.’ And we thought this showed that he had the same experiences that he would have on site.”

To learn more about the technology, Commerford provided the following contact: Doug Rufino, PSG Thermo Fisher Technical Operations Senior Director; email: douglas.rufino@thermofisher.com; phone (919) 452-3136.

[RELATED: How does your inspection rate compare to that of your peers? If you want to find out, get your Free Benchmark Report.]