The mini variety show actually began a year earlier with Pee-Wee's first appearance on Late Night. He would entertain with toys, singing, the Pee-Wee dance, and messing with students in a cafeteria video. He would later attribute the success of his stage show, which sold out Carnegie Hall in 1984, to being a guest on Letterman. Two of those appearances were on Halloween and almost started a tradition.
Pee-Wee would make a couple more appearances on Late Night before landing his Saturday Morning show but unfortuenately the Halloween tradition didn't last. Watching these videos now makes me wish it did last a few more years, but I guess once you hit the big time you don't need to do sideshow gags on talk shows anymore.