Polyethylene oxide (PEO)-based polymer electrolytes have been considered as the benchmark for developing solid-state battery systems since the 1970s. However, the limited oxidation stability (<4 V) of the ether oxygen in PEO's polymer structure renders it incompatible with high-voltage (>4 V) cathodes, which restricts further improvements in the energy density of lithium (Li) or sodium (Na) metal batteries. In this work, we develop a novel type of polymer in a low-concentrated electrolyte system, Na(TEGDME)4.1(PEO)1.0 (mole ratio of NaFSI:TEGDME:PEO = 1:4.1:1, calculated based on NaFSI mole). For a comparison, a liquid baseline electrolyte, 1 m NaTFSI in tetraglyme (TEGDME) (NaFSI:TEGDME = 1:4.1), were selected to compare with new electrolyte in term of redox stability, thermal stability, and charge-discharge performance. The hybrid gel electrolyte, Na(TEGDME)4.1(PEO)1.0, significantly enhanced oxidation stability by raising the onset oxidation potential from 4.0 to 4.2 V in comparing to non-PEO electrolyte. Additionally, the thermal stability of the gel polymer electrolyte was dramatically higher than that of the liquid electrolyte, with the onset temperature increased by over 75 °C relative to the evaporation temperature of the liquid electrolyte, which begins around 140 °C. The charge–discharge curves of sodium half-cells indicates that the gel polymer electrolyte exhibited a higher capacity compared to non-PEO electrolyte.