General requirements for regulated qualifications
Fitness for purpose (Condition D1)
1. A qualification provides information about the capability of an individual in a particular field. TLM's policy is to produce qualifications that fairly reflect a learner's competence in an area of study and/or employment. Such qualifications will have the following attributes, policies and procedures designed to reduce the risk in relation to fitness for purpose. (Condition D1.2)
Validity - Validity is about the extent to which a qualification reflects a learner's competence in the defined field. This requires the assessment criteria and methods to match activities that would be typically representative of performance. This in turn includes the context in which the assessment takes place. In cases of practical competence in a work environment, at least part of the assessment should be made in the context of the typical practical environment. Isolated skills tests on their own are of limited validity in representing competence although they might be used to supplement more representative methods. The first part of the design of a qualification will be to check that the learning outcomes are representative of competence in the field and that the assessment criteria will be truly indicative, when met, that the learning outcomes will have been achieved. This will ensure that the foundation of the qualification is valid. Assessment methods will then be implemented that directly relate to the assessment criteria with as few intermediary steps as practically possible. The context for assessment will be typical of the contexts required in a normal working environment in the field that is the subject of the assessment.
Reliability - TLM has detailed procedures for ensuring that the qualifications it provides reliably reflect the competence of the candidate at clearly defined levels of attainment and across a specified volume of work. These procedures are described in the sections related to design of assessment grading and delivery.
Comparability - TLM adopts national and European systems (eg the RQF, CQFW, EQF) for its accredited qualifications that set out the levels and size of qualifications. These systems are taken up widely by many other awarding organisations and provide a means of ensuring comparability across qualifications from different sources and between qualifications of similar type. TLM's moderation and evidence collection procedures including feedback to assessors and comparisons across moderators ensure that there is comparability internally and externally when used in conjunction with the wider national and international frameworks.
Manageability - TLM has developed a cloud based IT system based on Open Source software to improve the manageability of its qualifications while lowering costs and reducing risk. These systems are currently the subject of 3 Transfer of innovation projects funded by the EU Lifelong Learning programme. Typically TLM's methods can reduce comparable qualification costs by more than 50% and also significantly reduce the staff overhead in managing assessment. Moderation methods feedback to assessors and act as a means of on-going staff development that is built into learning practice. This further improves manageability and lowers associated costs.
Minimising Bias - TLM has a number of practical strategies to minimise bias. There is a comprehensive Equal Opportunities Policy, qualifications are independent of purchasing particular technology products from particular suppliers and in most cases there are free tools provided to aid management and administration. There is a policy for Reasonable Adjustments and monitoring demonstrates approximately equal take up of qualifications by gender.
2. Conflicts between aspects of fitness for purpose (Condition D1.3)
The main constraint on fitness for purpose is the cost-benefit equation. Establishing systems for ensuring validity, reliability, comparability and bias come at a cost which is broadly manageability. The TLM design process adopted a zero based view and has prioritised risk in terms of the regulatory conditions against manageability and costs. This process has two key dimensions that are part of the innovation. The first is in the use of cloud based technologies together with an analysis of costs in the supply chain. Wherever possible costs have been reduced eg in the use of open source software, the use of on-line feedback and automated moderation systems and publishing on demand, while maintaining a minimum standard in evidence collection and verification. The second is to develop the expertise and consistency of assessors rather than fixing errors in outputs from learners. This is similar to the method adopted by Deming in revolutionising the Japanese manufacturing industry. The principle is to invest in the people and systems to reduce the risk to quality in the outcomes. The internet and communication systems are a key enabler of this strategy. For this reason new assessors will have 100% of their work checked. They will be provided with feedback on their assessments and as they become more consistent the monitoring will be reduced. High risk qualifications will be monitored more frequently and with greater scrutiny than low risk. This helps strike a balance between the main conflict of cost and quality since the cost of quality assurance is minimised through targeting and making the best use of existing resources.
3. Accessibility of qualifications (Condition D2)
TLM has a comprehensive policy for equality of opportunity covering the accessibility of qualifications and this is monitored throughout the year by a named Director responsible for equality. A report is provided on an annual basis to the Board. If as a result of monitoring and review or any incident arising, an aspect of feature of the qualifications is judged to be disadvantageous or a risk of being disadvantageous to any learners, the qualification will be modified as a result to achieve fairness.
4. Reviewing approach (Condition D3)
TLM officers meet regularly to discuss qualifications and their operation on a day to day basis. There is a formal review on an annual basis and it is related to the customer care policy and policies and procedures for qualifications development. The evidence base for these reviews is primarily feedback and comments from users through informal and formal meetings. All centres have an annual centre visit where views and information can be gathered using on-line systems. There is a permanently accessible customer satisfaction questionnaire on the web site and data collected in the database and through training sessions is all used. If at any time there is objective evidence that an approach requires changing, this will be implemented taking into account any risk associated with the change that could in itself have an Adverse Effect. If there is any evidence in the current approach that any part of the approach has an Adverse Effect or has significant risk of resulting in an Adverse Effect. the approach will be modified again taking account of such risk associated with that change. The over-riding principle is that learners should not be disadvantaged. TLM has a good relationship with a selection of schools that help provide review data. A baseline testing project will ensure that 100s of thousands of learners will have been assessed using the on-line technologies prior to large numbers with the regulated qualifications. This stress tests the technologies so that we can improve their efficiency and reliability in a lower risk context. We will continue to obtain feedback from users through direct dialogue as well as metrics from the web site to ensure that all TLM qualifications are achieving their objectives. Also refer to the design strategy.
5. Responding to enquiries and complaints (Condition D4)
TLM operates a comprehensive set of policies and procedures for responding to enquiries and there is a permanently available complaints form available from the web site. All complaints will be treated seriously and will be acknowledged within 24 hours of receipt. The action taken will depend on the subject matter of the complaint. If the complaint is related to the award of qualifications it will be dealt with in accordance with the appeals procedure. If the complaint is related to the behaviour of a member of TLM staff, it will be investigated internally using the company disciplinary procedures. The outcomes of investigations of complaints will be fed back to the complainant subject to any over-riding need for confidentiality. In all cases TLM staff are expected to deal with complaints in a calm and even handed manner.
6. Compliance of qualifications with Regulatory Documents (Condition D5)
The design of TLM qualifications including unit design, rules of combination, and design of assessment grading and delivery, are based on the associated Regulatory Documents and checked to ensure full compliance. These take into account elements of good practice that are articulated as well as specific requirements. An example is the adoption of a risk based approach to qualifications in keeping with the Regulators risk based approach to Regulation.
7. Compliance of third party units with Regulatory Documents (Condition D6)
TLM's policy is to only use units, rules of combination as part of accredited qualifications that fully meet all the conditions set out in Regulatory Documents.
8. Management of withdrawal of qualifications (Condition D7)
The primary principle is to protect the interests of learners when any qualification is withdrawn for whatever the reason. TLM will provide adequate notice before withdrawing a qualification and it will inform the Regulators prior to informing end users. No learner who has embarked on that qualification will be unable to complete it given reasonable timescales typical of those need by the majority of candidates. The only over-riding factor to this policy is if instructed by the regulators to withdraw the qualification to a different timescale.
The withdrawal procedure is to give adequate notice with a deadline for the receipt of final assessment evidence. After this deadline the technology for receiving assessment grades under-pinning the certification process will be unavailable. All certificates awarded before that deadline will be available for authentication against the TLM database of qualifications. (Condition D7.4) Learner interests will be protected by allowing sufficient time from the announcement of the withdrawal to enable completion of any units of work that have been started. This policy will be modified in the light of any notice served by the Regulators to do so.
9. Information for teachers. (Condition D8)
It is TLM's policy to provide guidance in relation to interpreting and providing evidence against the assessment criteria with the amount of guidance being determined with the risk associated with the assessment. In subject matter that rapidly changes such as digital technologies, more detailed guidance is provided than in subject matter that is long established and very much well understood by the assessors. An example where guidance is necessary is in understanding open systems and open source products. Many professionals in the IT sector have never been taught about these increasingly important aspects and the guidance and feedback from moderation/verification provide an opportunity for CPD that would otherwise not occur. Where an examination is used, a sample exam will be provided and teachers encouraged to use it diagnostically and to provide feedback to TLM on any improvements that they think can be made from their own field testing.
