The Broad Street Synagogue (later known as Temple Beth-El) is located at 688 Broad Street in Providence. According to Wikipedia, the Jews of Providence who founded Temple Beth-El were predominantly Ashkenazi Jews from German-speaking areas. The majority of the early congregants were immigrants from Germany, the Netherlands, Hungary, and Poland. If I had to guess, I would say this is the synagogue where Houdini said his final Kaddish.
The building survives but is currently vacant. In 2020 it was named as one of named one of Providence's "Most Endangered Buildings" by the Providence Preservation Society.
Another candidate is The Sons of Jacob Synagogue located at 24 Douglas Avenue in the Smith Hill neighborhood of Providence. Wikipedia states that the congregation which built it was formed in 1896 by Orthodox Jews who immigrated from Russia. The Rhode Island Jewish Museum is currently running a campaign to restore the building and "into a center that curates the immigrant experience."
Like I said, I didn't do a thorough search for synagogues in Providence in 1926, so it's possible Houdini visited another one entirely. It's also possible he skipped the ritual this year, but I don't think that's very likely. He was devoted to both his parents and he loved a good ritual!
From the review of his opening night show, we know he had instructed an audience member to meet him at the theater at 10AM to be "disillusioned" about the medium Keeler. So I suspect Houdini started his day with his synagogue visit and remembrance of his father.