Some people need to learn lessons the hard way, by doing things the same as they have always done and experiencing the hardships for themselves.

But you don’t have to do this.  Learn these helpful commissioning lessons based on my commissioning experience. Through my commissioning experience, I have learned many lessons on what works and what does not in order to successfully commission mega projects.  Learn from my successes and from my mistakes to give yourself the best chance of success on your commissioning project.

Commissioning Team

The people on your team make all the difference. This is a common phrase – that budgets and schedules don’t make projects successful – people make projects successful.  And it is not just the skill sets that each person has – these are critically important, but don’t make the person successful.  When working in a team, each individual attitude is the most important thing.  If people have the right attitude and can work together (with the right skill sets), commissioning can be successful.  One “bad apple” on the commissioning team can ruin the dynamics of the entire team.  When a “bad apple” is noticed, it is important to address the situation immediately, which may include removing the individualt from the project.  It does not take long for one bad attitude to infect others on the team.

The commissioning team plays an important role early in the project.  The commissioning team has the end in mind and can help to bring that focus early in the project, to set the end of the project up for success.  The commissioning team’s early involvement can help shape the project and point it in the right direction.  Without this, people at the beginning of the project, who may not even be around for the end of the project, are making decisions that they will not be accountable for on behalf of others.

It is important that there is continuity in the commissioning team members, particularly the commissioning manager and commissioning leads.  The knowledge these individuals gain as the project progresses is invaluable to later activities.  A new individual can certainly join the project and pick up where someone left off, but they won’t have all the project historical information that will be needed should problems be encountered later in the project.  It is imperative that the knowledge gained on the project stays with the project.

The extended commissioning team requires diligent effort to engage and maintain relationships.  A lot changes as the project progresses and the pace of change increases as commissioning ramps up.  Groups that have been responsible for making decisions on the project may have difficulty letting go and allowing the commissioning team to make decisions.  The priorities of the project will shift as it progresses, and it is important to keep priorities aligned amongst all stakeholders.  Competing priorities will only make commissioning more difficult.  Strong leadership is critical for this aspect.

Learn more about team dysfunctions

Plan Commissioning Like a Master

Commissioning is fast paced.  Schedule and budget can easily be impacted by small issues during commissioning.  Therefore it is critically important to prepare for commissioning in advance.  Many months of planning is required in advance of the commissioning team mobilizing to site in order that commissioning plans are in place and ready to execute.

Now here is the biggest challenge – everyone on the project will want to optimize all the commissioning activities, with the assumption that everything will go perfect.  But it never does – and it can’t be expected to.  When energizing systems for the very first time, there will always be unexpected results, such as equipment failure (infant mortality) or unexpected operating results (equipment operating too hot, etc).  These unknowns need to be planned for.

But how do you plan for an unknown event?  A properly sequenced and scheduled commissioning plan will always have a contingency plan.  For example, let’s assume there is an upcoming major 2-week holiday where nobody will be on-site.  It would not be appropriate to plan a certain set of commissioning tasks to be completed the day before the holiday.  What if testing is delayed and can’t be fully completed before the holiday break?  The two options would be to ask a subset of the team to remain on-site and finish testing through the holiday break, or to leave a partially tested system in place during the holiday break for the operations team to monitor, who are likely at reduced staff during the break.  Neither of these situations are good, and the set of commissioning tasks should not be started before the holiday break if they cannot confidently be completed beforehand.  A better sequence is to plan other testing activities before the break, and plan to complete the more critical testing after the break.

Contingencies can exist on a smaller scale (such as the example above) or on a larger scale.  When planning for commissioning, the startup sequence should consider what would be the case when major subsystem components are delayed during construction.  What could the commissioning team continue testing in the absence of certain equipment?  In a lot of cases there is not much you can do.  But scenarios should be considered to adjust the commissioning sequence should there be delays.

Besides the unexpected issues that could be encountered, the commissioning teams need to prepare all plans, procedures, and checklists in advance of commissioning in order that all plans and procedures are in place to execute the well-thought-out commissioning plan.

Factory Acceptance Testing is Critical to Success

Factory Acceptance Testing is not just something that needs to be done when equipment is purchased.  It is an important part of the commissioning process.  Poorly executed FAT will only defer problems to sites where they are more costly to address and have a greater impact on critical path activities.

This may be the hardest lesson to learn the hard way – not properly planning and executing FAT for automation systems.  If the FAT does not include the PLC logic design, this will create a huge mess at the end of the project while you are trying to complete the automation design and make the systems work.  Automation cannot be an after-thought and must be properly planned and tested during FAT.

Factory Acceptance Testing needs to be part of the overall testing strategy of the project.  In many cases, the hardware and the software will only be integrated together for the first time at site due to the design taking place in parallel with procurement.  Instead, the testing strategy needs to include the software as part of the FAT.  This is known as an integrated FAT.  Solving all the errors and issues with the software in the factory with the actual hardware will significantly reduce the risks of on-site testing.

This does require that the software design must be advanced in order to be complete for FAT.  This can be difficult if it is a brand new design.  For existing designs that are modified, this is easier to do.  But for a brand new design with new equipment and new requirements, a lot of the electrical system design needs to be complete prior to the automation design taking place.

However, just because it is difficult, doesn’t mean that it shouldn’t be done.  Proper upfront planning that includes the necessary design and procurement coordination is necessary to see the project up for success at the beginning.  Trying to take shortcuts and defer the software design and testing to site will only add more time and cost to the project.

It is too tempting to complete the design in parallel with the procurement.  Procurement can often require longer lead times, and to someone who wants to learn this lesson the hard way, it would make sense to do the design during a 6-month leadtime for equipment.  But deferring the software testing to later in the project, you are just deferring the issues to later in the project.  Properly plan the upfront activities of the project to ensure later success.

I have been part of projects that have taken both approaches, and without exception, the projects that focussed on the software integration earlier in the project realized significant cost and schedule savings by planning this way.  I’m serious, projects saved years of schedule by taking an upfront approach to FAT.  I can think of two examples where the software was deferred to the end of the project which added years (yes, that’s plural) to the project schedule.  And with additional time for the project comes additional cost.

Learn this lesson now by reading this, or learn the hard way in a few years when your systems are not working at site and the owner is forced to accept a sub-par design and integration that is just barely made to work.

Solid Mechanical Completions Streamlines the Handover Process

The logistics of each mechanical completion can be complex.

On their own, each mechanical completion can be straightforward – a bunch of items to complete installation and inspection for, to be completed with paperwork on a certain date.  But add multiple mechanical completions that are moving targets and milestones not being met, and handover can become quite complicated.  How are systems commissioned if not available per the agreed-to sequence?

This is a lesson I have had to learn the hard way.  An upfront definition of each mechanical completion is imperative.  At the beginning of the project when it is being systematized, each mechanical completion needs to be defined.  Each mechanical completion sets the expectations of what is required and when it is required to align with other commissioning activities.  The mechanical completions set expectations between the construction and commissioning groups.  How can the construction group understand the needs and requirements of the commissioning team without a clear set of documents defining what they need to do?

All boundary conditions are defined in the mechanical completions since construction and commissioning work will certainly overlap within parts of the project to optimize the project schedule.  The mechanical completion documents define which areas are under test, and therefore under the care/custody/control of the commissioning team, while other areas are still under construction and under the care/custody/control of the construction team.  This clearly marked line-in-the-sand aligns expectations between both groups.

The goal of each mechanical completion and handover is zero snags.  This is a great goal to have but is rarely achieved.  The next section discusses how to strive for this during construction.

Construction Quality Will Make or Break the Project

If the construction process is managed well, that sets the commissioning team up for success.  But if not managed well, commissioning can be nothing but problems.  One of the biggest success factors during commissioning is the quality of the systems received from the construction group.  It is therefore critical that a properly managed Quality Management System (QMS) be implemented by the construction team.

A properly implemented QMS will define the contractor’s Quality Control (QC) role and the Quality Assurance (QA) role, often by the owner or a group delegated by the owner.  Both QC and QA are critical during construction to achieve the goal of no snags.

The contract defines the systems to build and the level of quality to achieve.  The contractor must therefore implement a quality control plan to ensure they are meeting these requirements.  The details of the contractor’s quality control is defined in Inspection Test Plans (ITPs).  Each ITP is written in advance and executed in the field, covering all aspects of the installed equipment.  This could include torquing bolts and witness striping, point-to-point checks to verify cables are pulled, glanded, and terminated correctly, or that no cables were damaged during installation by performing megger testing on all cables.

The contractor is required to inspect the work per their QC documents to ensure it meets contract requirements.

The owner will then perform QA on the contractor’s work.  Let’s assume the owner has hired a consultant to perform this QA role.  The consultant’s QA role is to ensure that the contractor is fulfilling their QC responsibilities.  The QA role is to audit the contractor’s on-site activities and documentation to confirm the work meets contract requirements.  This is not strictly a paperwork activity – boots-on-the-ground are required to visually look at the contractor’s work.  The QA role is not to witness or inspect 100% of the contractor’s work, but to ensure the agreed to ITPs and QC plans are being followed.  As an example, the contractor is responsible to do 100% point-to-point checks, to confirm cables are installed properly.  The consultant may choose to witness the first portion of how the contractor is doing the tests, but wouldn’t witness 100% of the point-to-point checks.  The consultant’s QA team would review the paperwork submitted by the contractor after point-to-point checks are complete to ensure all ITPs are performed and that all points pass.

The goal of handover to construction to commissioning is no snags – that the systems are able to be energized and tested with minimal errors in the installation.  This is the goal to strive for, but invariably things get missed.  A few small snags is not a big deal – a rolled wire can easily be swapped.  But when installations are deficient, cannot even be tested, and must be turned back over to the construction team, this is the sign of a poorly implemented QMS with minimal or no QC/QA.

In addition to quality, meeting milestone dates is equally important.  Each mechanical completion defines the scope of handover, and equally important defines when the handover is to occur.  The sequence of mechanical completions is important since this is how the system will be built-up and tested, which the order is rarely flexible.  For example, the power systems are needed prior to testing of the pump systems, since the pump won’t be much use without power to turn it.  The dates defined by the commissioning team to the construction team are critical to the success of the startup of the plant.

Execute Commissioning With Precision

On-site commissioning is a busy time.  Activities are moving at a rapid pace, with multiple areas under test at the same time.  All activities need to take place at the right time for everything to come together as one at the end of the project.

As mentioned earlier, the commissioning team needs to be ready for this fast pace in advance, since there will be no time to prepare during testing.  If things get behind, there will be minimal opportunity to catch up, and the only way to catch up may be evenings and weekends.  This is not a good situation for anyone, as this cannot be sustained for a long duration.  To accommododate this, catch-up days should be built into the schedule.  This could include 5 days per week testing, with no testing planned for Saturdays.  Saturday can be used if required as a catch-up day, but can typically be planned as a day off assuming the 5 days of testing went well.  I never plan for testing on Sundays, everyone needs at least one day off per week.  If situations do require 7 day per week testing, then multiple people should be involved, in order that everyone gets at least one day off per week.  Working the team to the bone is not a good way to plan safe execution of commissioning.

Prior to commissioning starting, the commissioning team should establish the support roles that are required.  Support may only be required part time, but agreements should be established in advance for all to be aware that when support is required, it may be required with short notice with priorities agreed to.  If you are not able to get the support you need until a week from next Tuesday, that is unlikely to be helpful during critical path commissioning activities.

Daily morning meetings are the best to plan the day’s activities and manage expectations.  Morning meetings are usually pretty early so that everyone can understand the plan and get out into the field to start executing early in order to get a full day of testing.  End of day meetings may also be requried, if there is a particular setup required prior to the next day’s activities.  End of day meetings are also a good idea to review any deficiencies to be addressed prior to the next day of testing.  This may require an evening shift to plan for the next day’s tests and address deficiencies, if schedule is the critical driver on the project.

One thing I can recommend is to pace yourself.  The commissioning phase will be fast and furious, and you need to maintain a high level of output for an extended time.  It is not reasonable to expect that you or the team will be able to work 14 hour days for an extended duration.  This may be required occasionally, but cannot be sustained.  Instead, plan for reasonable work hours with contingency plans if things do not work out as planned.  There will be more delays to the project if you burn the team out or worse cause accidents due to unsafe working conditions.  Plan for the duration so that you know it can be achieved safely and accurately.

Precise Schedule Management Pulls it All Together

Full time schedule management will be required to support the commissioning team.  The schedule will change by the hour and require constant updates.

However, it is not a good idea to include every commissioning detail in the schedule.  Managing commissioning by the hour or even by the day in the schedule will be an impossible task – you will spend all your time updating the schedule.

Instead, include block activities in the schedule that represent an outcome.  For example, if you have 2 weeks of loop checks to perform, include a 2-week activity to represent this.  Include the details of which panel is being tested each day outside of the schedule.  It is not important which panel is tested each day, just that all are tested within the 2-week duration.  That way, as things change, the schedule does not need updating.  Loop check progress can be tracked outside the schedule, and the schedule progresses once the 2-week activity is met.

The daily details are best planned and tracked in the morning meeting.  Not everything needs to be prescribed in advance, as in which panel is tested next Tuesday – that can be determined in the morning meeting as the work progresses, and the progress monitored to ensure it is completed within the 2-week duration in the schedule.

Depending on how you have planned the work, you can include only the block activities that have dependencies on others, to ensure the work is progressing per the plan.  This won’t burden the schedule with every detail and will make it more manageable.

And for work that is not progressing within the planned duration in the schedule, contingency days can be used for this.  If an additional day is required to complete loop checks, the team can use a Saturday for this.  This assumes that every Saturday is not used to catch up, that you have a good plan to start with, and that if Saturdays are required, it is not the same crew that has to work each Saturday.

Schedule becomes critical to manage for each mechanical completion.  The construction team will be tracking progress of construction activities, and needs to provide regular feedback to the commissioning team on their ability to meet milestones.  Based on productivity, it should be established if the work is progressing satisfactorily to meet dates.  If it is not, recovery plans need to be put in place, or impacts to the commissioning sequence determined.  It is not appropriate for the construction team to notify that they will be late days before a critical milestone – this needs to be communicated well in advance to evaluate options and determine recovery plans.

Schedule will be scrutinized the most during commissioning.  I’m not sure why this is, other than everyone starts to pay more attention to the completion of the project.  The construction schedule will be monitored and there may be delays, but scrutiny becomes even greater during commissioning and delays are evaluated even more.  With this, schedule reporting is extremely important in order that all stakeholders are aware of the commissioning progress and aware of any schedule impacts.  The more you communicate about schedule, the easier it will be to manage expectations.  Regular schedule forecasts need to be distributed at least weekly during commissioning.  A forecast of risks should be included and potential impacts to schedule included in order to get ahead of the potential issues.  Stakeholders will appreciate that you are looking ahead and have a plan should a risk turn into an issue.

Contracts Are Essential During Commissioning, But Can’t Slow the Process

The contract documents define the rules of the game which all parties must follow, including the commissioning team.  The commissioning team needs to be aware of all aspects of the contract, especially the technical specification. It will cause issues if the commissioning team is requesting things that are not part of the contract without going through the proper change management processes. The commissioning team may also be requested to provide input to parts of the contract such as achievement of payment milestones in order for the commercial group to evaluate payment to the contractor.

When you first join the commissioning team, the first documents to find are the contract documents to learn and understand all aspects of the project. Only understanding the rules for the game will you be able to move forward with commissioning activities.

The contract defines the responsibilities of the contractor, but also defines the responsibilities of the owner, including the commissioning team. All groups need to do their part to make the project successful.

The commissioning team needs to be very aware of the contract, but cannot be responsible for execution of the change management process.  The commissioning team will focus on the technical items within the bounds of the contract.  The commercial topics need to be kept separate from the technical discussion and addressed by the separate commercial team.  If the commercial discussion is mixed with the technical discussion, this will slow commissioning progress.  The entire team needs to focus on completing the job and dealing with the commercial issues through other channels.

System Startup is the Best Part of Commissioning

The operations team is an important group to be part of the commissioning team, especially during startup.  As new equipment is brought online, it will be interfaced with existing equipment (in the case of a brownfield site).  As new equipment starts to impact in-service equipment, this needs to be closely coordinated with the operating team.

Operational experience is a huge benefit to any of the commissioning team members.  Understanding how the systems are operated and maintained and being familiar with the challenges that the operations team will be faced with helps the commissioning team to align expectations with operators and get ahead of the operational issues during startup.  Capital projects work is very different from daily operations work, and having an understanding of both perspectives is very beneficial.  Having an operational understanding allows the commissioning team to understand the owner’s operating team and how best to address their needs to set them up with the best chance of success after the project is complete.

System startup will take longer than expected and needs to be planned and coordinated to align with other operational activities.  Because of this, an activity that would normally take 1 day may need to be planned over 3 days to allow other plant processes to proceed.  Or possibly planned overnight when there are less demands on the systems.  Discussions with the operations teams need to start well in advance of commissioning to align expectations and plan the work accordingly.

Some plant operations are seasonally restricted and can only be done at certain times of the year.  In Canada, the cold weather is always a restriction, and working with fluids outdoors can be challenging or impossible.  These seasonal restrictions need to be considered when planning your startup activities.

Other startups need to be closely coordinated with neighbouring utilities or neighbouring jurisdictions and cannot occur at a moment’s notice.  These need to be discussed and planned in advance, sometimes a year or more to plan activities, in order that they can take place at the proper time.

Safety is always an important aspect during startup, as starting equipment for the first time on-site can be dangerous.  Safety must be considered and all safety precautions put in place during plant startup.

And don’t forget that once the equipment is started, it needs to be continually monitored until confirmed to be operating in a stable manner.  In some cases, not all automated monitoring and response systems are enabled at this stage of the project.  In this case, 24 hour on-site monitoring may be required to alert others should there be a problem with the equipment.

And lastly, celebrate successful startup activities.  These are exciting days when equipment is started for the first time and everyone’s hard work finally pays off.  It may be just the commissioning team that is part of the startup, but be sure to send notice to all on the project that it was a successful day and the new equipment is operating as expected.

Operational Readiness Ensures Long Term Success

The commissioning team plays a role in operational readiness.  Since the commissioning team is the last group working on the last phase of the project, they play a role in helping the operations team learn and understand the new systems.  The best way to do this is to have the operating personnel participate in commissioning.  This allows them to work with the systems and be part of the troubleshooting, so they can ask the equipment experts any questions that they have and become familiar with the new systems.

This works out well, since operational input is critical to the commissioning process.

The commissioning team will also contribute to the final documentation required for operations.  Operating procedures will be created as part of the commissioning process, and these can be used to create the final Standard Operating Procedures (SOPs) that will be used by operations.  As well, any final settings or red-line markups will be provided by the commissioning team to document the as-left state of the equipment.  These settings and markups will be included in final documentation for the operating team to use.

Although the commissioning team may not be responsible for all operational readiness items, they are a large contributor to the process.

Learn more about Operational Readiness