The Gunners went behind after a Josip Pivaric shot was saved by keeper David Ospina but rebounded in off Alex Oxlade-Chamberlain.
Giroud was then sent-off for a second booking when he fouled Ivo Pinto before Junior Fernandes headed in for Dinamo.
Theo Walcott superbly slotted in to give Arsenal hope but the home side nervously held on for the win.
Walcott had scored a hat-trick at the same Maksimir Stadium for England against Croatia in 2008 but, despite pulling one back, he could not repeat such heroics.
Dinamo held on to extend their unbeaten run to 42 games as they tasted victory over an English top-flight side for the first time in the Champions League.
France striker Olivier Giroud has been criticised for not being clinical enough in front of goal and he only added weight to that argument against the Croatian champions.
He had a free header from a corner palmed away by Dinamo keeper Eduardo and, when another nodded effort came off the post, Giroud failed to sidefoot in the rebound as the home stopper smothered his point-blank shot.
The Arsenal forward’s frustration got the better of him when he was booked for dissent after his reaction to the referee blowing for a foul against him.
It proved crucial as he was then sent-off for a second bookable offence when he caught Pinto with a late challenge after 40 minutes.
Arsenal lost their opening game of last season’s Champions League when they were beaten 2-0 by Borussia Dortmund, but recovered in a group containing Galatasaray and Anderlecht.
This time they face a more difficult task to make it past the group stages for a 13th consecutive season.
The Gunners host Olympiakos next before home and away games against German giants Bayern Munich.
Bayern are one of the favourites to win the competition and beat Olympiakos 3-0 as they moved to the top of Group F.
Dinamo might have been the home team but they employed the tried and tested philosophy of playing on the counter-attack to try to beat Arsenal.
The Croatian champions soaked up Arsenal’s pressure and, after the Gunners failed to take their chances, took the lead.
Arsenal manager Arsene Wenger made six changes from the side which beat Stoke, including dropping keeper Petr Cech and holding midfielder Francis Coquelin to the bench.
Wenger might have had Saturday’s game against Chelsea, who beat Maccabi Tel Aviv in their opening Champions League game, in mind but the move backfired. -BBC