I've been wanting to write about this for a while. So, without having listened to the piece in some months, and having read nothing scholarly on the subject, I'd like to address the oboe cadenza in the first movement of Beethoven's fifth symphony.
Despite it's brevity, the cadenza's importance cannot be overlooked when listening to this piece. There are many potential reasons why the maestro included it, and it's impact is felt on many intellectual levels. I've given the matter some thought, and I've come up with my own short list of potential reasons for the cadenza's inclusion.
1) It's a form of musical joke. It's like the wannabe soloist who strikes out on a third out-chorus when the choirmaster has shortened the piece to two, or the lonely Protestant who adds extra lines to The Lord's Prayer at a mass. The clueless individual is so into his part that he keeps going after everyone else has stopped, with comical results. And just like the former group member who's become an isolated object of ridicule, the oboe cadenza's first notes are strong and logically consistent with the music that preceeded it, but then become progressively confused as it fades into silence and conformity.
It's worth noting that after his first symphony, in which he called the third movement a "menuetto" (standard fare for the time it was written), Beethoven used the term "scherzo" or joke for the third movement of all of his symphonies (except the ninth, of course, in which the second and third movements are reversed in order). Meaning (possibly) that he was not above the idea of injecting humour into serious music, or (again possibly) that he felt himself above following the established rules of composition etiquette.
2) Even if it is a joke, it makes a serious point. The first movement of this symphony is a monolithic display of the severe power of conformity. It turns a chamber orchestra into a unified force to sear it's simple melody and variations into the mind of the listener. The cadenza is a short, quietly juxtaposed off-melody, and can be seen to symbolize the lonely individual following his own path away from the crowd (much like Beethoven himself).
3) Beethoven wanted to create a sonata form movement that even the most basically educated music listener could grasp and understand. Sonata form is fairly simple to begin with: it is divided into three sections, being the exposition, the development, and the recapitulation. The exposition consists of two juxtaposed musical ideas, a strong theme and a gentle theme, presented in their most basic melodic forms and almost always in the strong/gentle order. In the development section the composer derives new music by applying techniques of variation to the two themes. In the recapitulation the composer re-presents the two themes along with some of the newly developed ideas, mounting to a crescendo and finale. However, in many examples of sonata form it is difficult for an inexperienced listener to discern where one section ends and the next begins.
Beethoven was a master of theme and variations, and no concept is more important in his work. His earliest composition, published while he was still a teen, is a theme a variations for piano. The first movement of the fifth symphony is a nearly perfect example of theme and variations, to it's very core. The strong theme consists of eight notes (which in themselves are a four note theme quickly followed by a four note variation) and a simple set of variations based on those eight notes. The gentle theme provides some relief by way of a longer melody designed to soften the dramatic, punchy effect of the strong theme. But even then, it is started by a six note variation of the core four notes from the strong theme, played by a solo horn so as to differentiate between strong and gentle.
Beethoven delineates between the exposition and the development sections is a unique way. When the strong and gentle themes have been fully presented, he stops the music completely and starts over again, repeating the two themes note for note. The effect is such that the beginning of the development section stands out, as it goes beyond what the listener has already heard.
To delineate between the development and recapitulation a different technique was required. The nature of development means that bald repetition could not be used. This is why the oboe cadenza comes in - it is used to clearly mark the boundary between the development and the recapitulation, much the same way the solo horn marks the difference between the strong theme and the gentle theme.
4) More than wanting to write a sonata form movement that was easy to follow, Beethoven wanted to write a sonata form movement that was textbook perfect, but for some reason he balked at the last moment. The oboe cadenza is an intentional anomaly in an otherwise perfect score. Perhaps he was superstitious, and didn't want to tempt fate by creating a work that was perfect, and therefore unsurpassable.