Along with other items like the Crimson Agate, Rime-Worn Fragments and Crystals are a new thing added to Genshin Impact in update 1.2. These new items are added as part of an event in-game. Rime-Worn Fragments and Crystals are part of Act I of The Chalk Prince and Dragon Event; which means that these are found in the Dragonspine region. You will not be able to earn these items outside the new regions, so start there.
Rime-Worn Fragments are the much more common drop of the two. Though you need to get them soon, they will cease dropping on 4:00 PM EST on January 6, 2021. You can get them as an uncommon drop for any enemy within the Dragonspire region. You won’t need to hunt down any particular beast to get them, so that’s good news. Rime-Worn Fragments are included in nearly every mob’s loot table. The trouble begins when you start having to hunt for the Crystals.
Getting Crystals involves heading into the newly spawning Snowstorms and hunting them down. This means you need to have various means to fight off the Sheer Cold effect. These can range from limited-time effects from food items, or for more permanent options, you can opt for the Warming Bottle. You need to gather 2 Flaming Flower Stamen, and 2 Starsilver to craft one.
Once you have that, you’re ready to begin.
Rime-Worn Crystals drop when you kill the larger enemies, so you need to hunt them down and take them out. So get out there and take out some mini-bosses to farm plenty of Rime-Worn Fragments and Crystals in Genshin Impact. These appear to fairly rare spawns, so you will always have more Fragments than Crystals, and that’s fine.