The Brazil forward was making his much-anticipated first appearance for PSG since joining them from Barcelona for a world-record £200m transfer fee.
And he crowned PSG's impressive second half show by finishing from close range eight minutes from time.
Neymar had earlier provided a delightful 62nd-minute pass from which Edinson Cavani struck to make it 2-0.
And the world's most expensive player had also been involved in the build-up to the opener in the 52nd minute, a Jordan Ikoko own-goal, as he attempted to play in Cavani.
Neymar started on the left of a front three also featuring Cavani and Angel Di Maria and was given considerable licence to roam.
The 25-year-old produced plenty of impressive moments in what will surely go down as a memorable start to his PSG career.
Neymar drove one shot over the crossbar but Marquinhos came the closest to breaking the first-half deadlock after his bullet header cannoned back off the crossbar.
Yannis Salibur went close for Guingamp, but the visitors were soon back on the front foot and it was no surprise when they took the lead - only how it came about.
Neymar's attempted pass to Cavani was intercepted by Pedro Rebocho, diverting the ball to Ikoko, who somehow managed to send it past his own goalkeeper and into the net.
Three minutes later Neymar slid in a wonderful pass for Cavani, who slotted past Karl-Johan Johnsson in the Guingamp goal.
It looked as if that might be his most notable contribution, with Johnsson saving well to deny him towards the end, but within seconds of that effort Neymar finished Cavani's cutback