000			04409ntm a2200337 i 4500
999			_c101215 _d101221
003			MY-KuUP
005			20251125110922.0
006			t||||fr|||| 000 0
007			ta
008			240902s20242024my a|||fr|||| 00| 0 eng d
020			_aTHE0009940 (Local) _qHardback
040			_aUMPSA _beng _cUMP _erda
090			_aFKOM .A23 2024 r Thesis
100	0		_aAbbas Saliimi Lokman, _eauthor.
245	1	0	_aModified word representation vector based scalar weight for contextual text classification / _cAbbas Saliimi Lokman
264		1	_aKuantan, Pahang : _bUMPSA, _c2024
264		4	_c© 2024
300			_axii, 120 pages : _billustration ; _30 cm. +30 cm. + _e1 CD-ROM
336			_2rdacontent _atext
337			_2rdamedia _aunmediated
338			_2rdacarrier _avolume
347			_2rda _atext file _bPDF
500			_aFaculty of Computing
502			_aThesis (Master of Science) -- Universiti Malaysia Pahang – 2022
504			_Includes bibliographical referencesIncludes bibliographical references
520	3		_aThis thesis investigates contextual text classification, which is the process of categorising textual data into different classes or categories based on its meaning within a given context. Central to this process is the representation of words through vectors for computational interpretation. Current practices employ Large Language Models (LLMs) to generate contextualised word representation vectors, achieved through pre-training the LLM on vast corpora that enables it to grasp intricate language patterns and context. For contextual text classification, the pre-trained LLM is further train on classificationspecific labeled data in a process called fine-tuning. Although this approach is currently considered the most optimal in the field, it poses a notable challenge due to the substantial demand for computing resources stemming from the vast number of trainable parameters in LLMs. Furthermore, although pre-trained LLMs can generate contextualised word representation vectors, they lack the flexibility to modify the semantic significance of these vectors outside of the LLM, necessitating fine-tuning for the modification of word vectors. To bridge this gap, a five-phase research methodology is structured to propose and evaluate an algorithm enabling the external modification of LLM-generated word vectors using scalar values as the focus weightage. To validate this algorithm, the modified word vectors are compared with original LLM-generated word vectors to evaluate their reflection of the intended context. In addition, a contextual text classification experiment is conducted using benchmarked datasets to assess the performance of the modified word vectors in the targeted classification task. For this experiment, the modified word vectors serve as input to train a Machine Learning (ML) model for the text classification process, aiming for the developed ML model to have a significantly smaller parameter count. This experiment aims to determine the effectiveness of the modified word vectors in contextual text classification tasks, utilizing a more computationally efficient approach. Based on the acquired results, the experiments reveal that the modified word vectors algorithm can effectively alter original LLM-generated word vectors to reflect intended contexts and can outperform baseline scores in contextual text classification tasks. Evaluation metrics including Accuracy, Precision, Recall, and F1 score are employed in the evaluation process, with Accuracy and F1 score serving as primary metrics. The evaluation showcases significant improvements, with the test ML model achieving a best accuracy score of 0.571, a 46% increase from the baseline, and a best F1 score of 0.727, a 30% increment from the baseline. Overall, this thesis presents five contributions: the proposed modified word vectors algorithm, the new contextual classification dataset named QCoC, the efficient question-type classifier based on the feed-forward neural network algorithm, the potential transferability of the presented work to other domains, and the practical implications of the presented work towards cases where computational resources are limited or costly.
610	2	0	_aFaculty of Computing _xDissertations
650		0	_aUniversities and colleges _xDissertations
650		0	_aTheses _xDissertations
942			_2lcc _cTHESIS