The all-solid-state lithium batteries using solid electrolytes and metallic lithium anode exhibit high energy-density and safety, and excellent ability to inhibit the growth of lithium dendrite and improve the intrinsic safety of batteries. Solid-state electrolytes as the key material have become one of the research hotspots for the past few years. The composite solid-state electrolytes obtained by dispersing inorganic fillers in the polymer matrix are the most promising electrdytes, because of their excellent mechanical and electrochemical properties. In the composite electrolytes, the interface between ceramic fillers and polymer matrixes exhibit a huge contribution to the ionic conductivity. However, the increase of the two-phase interface has a percolation threshold with the content increase of fillers. Therefore, it is important to optimize the percolation structure, dimension and arrangement of the fillers for the further improvement of the ionic conductivity. Herein, this paper summarizes the characteristics and Li-ion conductive mechanism of several solid electrolytes and composite solid electrolytes, analyzes different research ideas of percolation structure, methods and the performance of the composite solid electrolytes based on different dimensional inorganic fillers. At last, the bottleneck of designing percolation structures with inorganic fillers and the goal of future researches are prospected.