PDFA Distillation via String Probability Queries

Research output: Other contributionScientific

Abstract

Probabilistic deterministic finite automata (PDFA) are discrete event systems modeling conditional probabilities over languages: Given an already seen sequence of tokens they return the probability of tokens of interest to appear next. These types of models have gained interest in the domain of explainable machine learning, where they are used as surrogate models for neural networks trained as language models. In this work we present an algorithm to distill PDFA from neural networks. Our algorithm is a derivative of the L# algorithm and capable of learning PDFA from a new type of query, in which the algorithm infers conditional probabilities from the probability of the queried string to occur. We show its effectiveness on a recent public dataset by distilling PDFA from a set of trained neural networks.
Original languageEnglish
Number of pages13
Publication statusPublished - 2024

Fingerprint

Dive into the research topics of 'PDFA Distillation via String Probability Queries'. Together they form a unique fingerprint.

Cite this