Islanding detection in distributed generation (DG) interfaced to the microgrid is a key criterion for safety and power quality of the power system. This study proposes a transient monitoring function (TMF)-based islanding detection technique, which distinguishes between various islanding and non-islanding conditions in an inverter-based DG. Initially, the voltage samples of each phase retrieved at the target DG location are processed through Fourier-Taylor transformation (FTT). The TMF is the difference between the estimated voltage samples regenerated from the FTT and the actual sample values. Finally, the TMF index for islanding detection is computed by combining each phase TMF. Based on simulations carried out in PSCAD/ EMTDC environment, the performance of the proposed method is examined in various critical conditions. Therefore, the proposed method provides generalised solution irrespective of active and reactive power mismatch and thus enhancing the nondetection zone. The proposed TMF-based anti-islanding relay is highly efficient and satisfies the speed criteria of the relaying function in performing the task.