Those critics, mostly Republicans, fired shots at the National Recovery Administration, or NRA. They made some jibes about the Civilian Conservation Corps, or CCC. And they quipped about the Works Progress Administration, or WPA.
Behind the jokes, those conservative critics of the New Deal had a point. They thought the proliferation of agencies - many of them with overlapping missions - blurred lines of authority and accountability and make efficiency much more difficult.
Flash forward 75 years and move the scene to Indiana, where the same criticism could be leveled at the Republican Party's approach to education administration.
The back-alley brawl between Indiana Superintendent of Public Instruction Glenda Ritz (on one side) and Gov. Mike Pence and the state Board of Education (on the other) has gained a great deal of attention.
Often lost in the analysis is just how crowded the field upon which the combatants have been tussling is - and how those crowded conditions may have helped lead to the conflict.
One catalyst that kicked the squabbling into overdrive was the creation of the Center for Education & Career Advancement, a new state agency that will have to struggle to find elbow room at the Statehouse.
With the creation of this new agency, Indiana now has a superintendent of public instruction, a department of education, a board of education, an education roundtable, a higher education commission and a partridge in a pear tree.
In some cases, the membership of these boards, roundtables, agencies, commissions and pear trees overlap, which further confuses things and makes it that much harder for anyone inside or outside the process to determine where one entity's responsibilities end and another's begin.
That has been one of Ritz's arguments.
She has argued that the proliferation of education-related state agencies, departments, boards and roundtables has created an organizational that resembles the tracks in the dirt left at the conclusion of a demolition derby.
The blurred lines of authority, the overlapping of missions and the sheer number of voices yapping in the board room or at the round table make it almost impossible to hold anyone in government accountable for anything related to education. She said that the Center for Education & Career Advancement was a straw dropped on an already overloaded camel's back.
Ritz has a point.
Members of the GOP should understand that, because the point she's making is the one they usually do.
Republicans pride themselves on being the party of small government, the ones who have a passion for efficiency and streamlined systems of accountability. Most of the time, they look for ways to scale back the size of state agencies.
But in the case of education in Indiana, they have piled one agency, board, roundtable and commission on top of another. In the process they've created an education system that is so muddled that it's impossible to tell what's going on, much less who is responsible for what.
I don't agree with much of what conservatives want to do in regard to education reform, but I've been around long enough to know that they are sincere in their desire to create an education infrastructure that is nimble enough to embrace effective innovation quickly and more responsive to the concerns and needs of students and parents.
My humble suggestion is that the state's Republican leadership could start the work of building such an infrastructure by honoring their historic principles and establishing such a system at the top.
They could lead by example.
If there were fewer agencies, boards, commissions and roundtables crowding the Statehouse - and fewer education experts, many of them self-proclaimed, struggling to find an audience - we Hoosier might find it easier to understand what we're fighting about.
Or, even better, we might find that we don't have as many reasons to fight as we thought we did.
John Krull is director of Franklin College's Pulliam School of Journalism, host of "No Limits" WFYI 90.1 FM Indianapolis and publisher of TheStatehouseFile.com, a news service powered by Franklin College journalism students and faculty.
Not far into the New Deal, Franklin Delano Roosevelt's critics had a lot of fun at his expense by cracking jokes about the "alphabet soup" he'd cooked up by creating a lot of government agencies known only by their acronyms.