Finite cell method (FCM) is an efficient numerical method for analysis of continuum structures with geometrical complexity, which uses the concept of virtual domain method. The use of this concept makes the mesh generation easier such that simple uniform meshes can be employed, and hence mesh refinement is not necessary for the problems like damage detection. Nevertheless, the FCM has not yet been used for damage detection of structures. In this paper, the FCM is proposed for structural damage identification. The identification of flaws based on dynamic response of the structure is investigated with the FCM and particle swarm optimization (PSO) algorithm. The FCM is utilized to model the flawed structures, while the PSO —as a well-known non-gradient-based optimization method— is employed for finding the flaw location. For the damage detection problem, the PSO explores the search space in order to detect the geometrical properties of flaws as well as the number of flaws by minimizing an error function in terms of sensor measurements. Four numerical examples including benchmark test cases are provided to demonstrate the capability of the FCM and the PSO algorithm in damage detection of plate structures.