The clock on his body is vulnerable to

Mirage status effect, which will cause temporary stun and open up his core.
Dealing enough damage to the clock core before he recovers will open up a permanent weak spot for the rest of the battle.
You have three chances to expose the core with Mirage before it becomes impossible to re-open. After breaking the clock, he becomes invulnerable to Mirage status.
Phase 1:
Loser will just do sword attacks. Break the clock during this time and force the next phase.
Phase 2:
The crystals around his shoulder will gain a rainbow tint and become targetable, and Loser will begin to affect the flow of time.
If the background is red, he will teleport around and attack quickly. If the background is blue, he will move slowly, but his attacks will do far more damage.
Phase 3:
Same as above, but all his attacks will have added effects, such as stun lightning bolts or fire patches on the arena.
At certain damage amounts, Loser will teleport into the background and put a sword in each corner of the platform, then go for a time freeze.
You can avoid the time freeze by watching the bubbles around the swords/the screen shaking and dodging right before the freeze effect activates.
If you are caught in it, you can break out of it the same way you can the Stun effect. During the time freeze, players must destroy at least one sword to avoid taking approximately too much damage.
When you destroy one sword, the area it was in is entirely safe from the attack, so if you can't destroy your own sword in time pray somebody else already has.
The next time freeze goes off a LOT faster than the previous one, and gives much less time to destroy swords with.