What is a Plugtests
TM Event? Plugtests refers to
event
where engineers get together to test the
interoperability of their implementations between
each
other.
Plugtests are part of the standardization
process.
The implementations tested may be prototypes
or products ready for commercialization
The events are open to every developers
The events are usually of short duration
(1-5 days)
They take place within the time frame of the
standards drafting or shortly after its
completion
What is the goal of a Plugtests TM Event? The aim is
Interoperability. This is what
standards are all about. Plugtests increase the
probability of achieving interoperability by
debugging the standard and companies’
implementations at an early stage.
What happens during a Plugtests
TM Event? Plugtests are for engineers who are implementing a
standard. They are not for people who just want
to see what the competition is working on.
Plugtests are also not trade shows. So care needs to
be taken that only engineers with implementations
attend.
It seems
that pretty sensitive information may be
revealed at Plugtests. Doesn’t this risk
outweigh the benefits? The fact that companies do go to these
events demonstrates that, at least for those
companies, risk outweighs benefits. We
put this question to the participants,
on a scale of 1
(risks outweighs benefits) to 5
(benefits
outweighs risks), the total average
score of about70 events
is 4.2.
Aren’t
interoperability events benefiting only small
companies? No. The
question assumes – wrongly – that a small
company has an implementation which is
inferior to that of a big company.
Aren’t
Plugtests benefiting only non-experienced
companies, i.e. companies that are just
starting to implement the standard, over
experienced companies? That’s
precisely the reason why the time-window for
such event is rather short, i.e. only as
long as the standard is still in the making.
As soon as there is a general understanding
on how to interpret a standard, there is no
more incentive for the experienced player to
participate in Plugtests and give newcomers
lessons for free.
Is a
Plugtests Event only for equipment manufacturers? Most of
the participants are equipment manufacturers
(vendors). But often operators are
experimenting with the standard as well,
writing their own implementations to get
a better understanding of the standard
and of potential problems.
What are
the BENEFITS of a Plugtests? The events
help to enhance the quality of a standard in
the making.
Plugtests find errors or ambiguities in
standards and thereby help improve the
quality of standards.
Fast
feedback into the standards process The
earlier bugs are discovered, the cheaper it
is to correct them. To start writing
conformance test specifications, the
standard has to be quite mature. Also, the
process of generating Abstract Test Suites
does take some time. Plugtests take place
early in the standards process and therefore
allow fast feedback on possible errors or
ambiguities in the specification. Engineers say that these events definitely
speed up the process of bringing products to
the market.
Increased
circle of participants - developers are
active in the process.
Close
interaction and cross fertilization between
the people drafting and those implementing
the standard
is beneficial for the quality of the
standard.
Plugtests
gauge the market acceptance of a standard.
Publicity
of Plugtests will raise awareness in the
market about the standard.
Debugging a
vendor’s implementation Testing
against products of other companies,
engineers find errors in their
implementation that they would not have
found otherwise, or only much later.
Meeting other developers familiar with the
standard By
making personal contact with developers of
other companies and getting to know them,
questions and answers that one doesn’t want
to post on a public mailing list can be
exchanged after the event.
Overall,
Plugtests save companies and the industry
lots of time and money Engineers rank the usefulness of these
events consistently very high. In addition
to discovering problems, these events allow
testing with a wide range of vendors in a
short amount of time. According to
participants, much time and money is
therefore saved as opposed to one-on-one
scheduling.
What do
Plugtests have to do with standardization? They
take place while the standard is still in
the making. As long as there are changes to
the standard, there will be interest in
Plugtests. Once the standard has settled,
participation at Plugtests may drop until
new certification releases are introduced.
In
contrast, interoperability testing as done
by e.g. test houses or network operators
comes later in the process, i.e. when the
implementation is about to be ready for the
mass market.
How can
Plugtests enhance the quality of
specifications? Plugtests take place early in the process,
namely while the standard is still being
drafted. They don’t guarantee
interoperability, but they increase the
probability of interworking. They enable the
identification of areas for improvements,
errors and ambiguities within the standard.
Don't well
written standards make Plugtests
superfluous? The
better standards are written with few
options, the easier it is to achieve
interoperability. Good standards don’t make
Plugtests superfluous, but good standards
necessitate fewer of them.
Are Plugtests
something new? No, Plugtests
have been around for many years, though
less in the telecommunications world.
They are also known as "Plugfests",
"Interoperability events", "Group tests"
and "IOP".
