The Weiss family landed in America on July 3, 1878, passing through the Castle Garden Emigrant Landing Depot in New York City (Ellis Island was not yet built). Yes, this means the first full day in America for the 4-year-old Ehrich Weiss was, appropriately enough, Independence Day.
There's no record of how the family spent the day, although it's likely they initially stayed at the home of Cecelia's sisters, Rose and Sally, who had immigrated to America in 1865-66 and lived in New York City. So there's little doubt young Ehrich got to experience some kind of Fourth of July celebrations on that day in 1878.
Ironically, the first record of how Houdini spent a Fourth of July comes from Russia. In 1903 Houdini and Bess traveled to Moscow where he famously escaped from the Siberian Transport Prison Van among other feats. They happened to be in Russia during the Fourth of July. So how did they celebrate this most American of all holidays in the land of the Czar? Houdini himself told readers in his New York Dramatic Mirror column:
Although yesterday was the Fourth of July in America, it is only June 21 here in Russia, but just the same our American Consul, Smith, invited all the American performers who were in Moscow to a celebration on the grounds of the consulate, over which waved a large American flag, while the Russians looked over the fence and wondered what the "always-in-a-hurry Americans" were up to. Among the guests were the Manhattan Quartette, Smith and Doretto, Weston of loop the hoopology, four of the Florida Girls, Miss Walcott, Mrs. Harry Houdini and myself. There were many speeches, but not a single firecracker or report of a gun was heard, and without these it did not seem at all like the Glorious Fourth to me.
The photo above shows that very celebration, and is the only photo of Houdini in Russia that I'm aware. (He can be seen leaning in smiling mid table on the right.)
A more traditional Independence Day celebration for Houdini was July 4, 1922. Harry and Bess attended a party at the home of Houdini's lawyer Bernard Ernst in Sea Cliff, Long Island. What happened that day Houdini would call "the most remarkable coincidence that ever happened to a mortal man", and he would share the story during his lectures on Spiritualism. In his 1924 book, A Magician Among the Spirits, he related the story in full.
[...] Not only are mediums alert to embrace every advantage offered by auto-suggestion but they also take advantage of every accidental occurrence. For instance, my greatest feat of mystery was performed in 1922 at Seacliffe, L. I., on the Fourth of July, at the home of Mr. B. M. L. Ernest. The children were waiting to set off their display of fireworks when it started to rain. The heavens fairly tore loose. Little Richard in his dismay turned to me and said:
"Can’t you make the rain stop?"
"Why certainly," I replied and raising my hands said appealingly, "Rain and Storm, I command you to stop."
This I repeated three times and, as if by miracle, within the next two minutes the rain stopped and the skies became clear. Toward the end of the display of fireworks the little fellow turned to me and with a peculiar gleam in his eyes said:
"Why, Mr. Houdini, it would have stopped raining anyway."
I knew I was risking my whole life's reputation with the youngster but I said:
"Is that so? I will show you."
Walking out in front I raised my hands suppliantly toward the heavens and with all the command and force I had in me called:
Houdini and Ernst in Sea Cliff.
"Listen to my voice, great Commander of the rain, and once more let the water flow to earth and allow the flowers and trees to bloom."
A chill came over me for as if in response to my command or the prayer of my words another downpour started, but despite the pleading of the children I refused to make it stop again. I was not taking any more chances.
May you have a rain free fabulous Fourth!