Trailing by 15 at half-time, the Rollers took a while to adjust the game while Japan flew out of the blocks courtesy of a home court advantage and rabid fan support from the crowd.
Brett Stibners (11 points, eight rebounds) was a shining light for the Rollers but a couple of his team mates found themselves in foul trouble.
While coach Craig Friday was keen to rotate through his entire roster, he was forced to adjust his line-ups to find the right combination but out of the main break, a mid-point line-up of Jannik Blair (two rebounds, one assist, two steals), Tom O'Neill-Thorne, Kim Robins (four points, two steals), Shaun Norris and Bill Latham (ten points) saw the pressure rise on Japan with a couple of consecutive stops giving the Rollers some momentum.
They also had their most productive quarter with 20 points for the term, trimming the deficit to ten at the last break and they continued the comeback early in the fourth period with Japan needing a time-out when the margin hit just six points.
Two more stops out of that time-out got the game back to a single possession which set up a thrilling finish as both teams battled for ascendancy.
Norris (16 points, nine assists) was clutch for Australia though with an elbow jumper before O'Neill-Thorne (21 points, four assists) connected on a huge and-one lay-up to give the Rollers a one-point victory.
"It was a great comeback through some adversity and challenges," said Rollers coach Craig Friday.
"In the opening game of the tournament, it was a small step forward."
Find the box score here.
The Rollers are back on court today with a double-header against Great Britain and Turkey before they play their final game over the weekend.
The Australian Rollers World Challenge Cup Schedule (all time AEST):
September 1: Australia vs. Great Britain at 11am
September 1: Australia vs. Turkey at 6.30pm
September 2: Final at 11.30am or 3.10pm