For the most part it's difficult for inventors to appreciate where their inventions will find their first market. Peter Drucker details this in an interview in Inc. magazine http://www.inc.com/magazine/19960515/2083_Printer_Friendly.h... which builds on his work in "Innovation and Entrepreneurship" http://www.amazon.com/Innovation-Entrepreneurship-Peter-F-Dr...
Drucker: The first entrepreneurial pitfall comes when the entrepreneur has to face the fact that the new product or service is not successful where he or she thought it would be but is successful in a totally different market. Many businesses disappear because the founder-entrepreneur insists that he or she knows better than the market.
Inc.: So, often the entrepreneur is actually succeeding but doesn’t realize it?
Drucker: No, it’s worse than that. He or she rejects success. You want examples? There are thousands of them, but one of the best is over 100 years old.
A man by the name of John Wesley Hyatt had invented the roller bearing. He made up his mind that it was just right for the axles of railroad freight cars. Railroads traditionally stuffed the wheels of their cars with rags soaked in oil to handle the friction. The railroads, however, were not ready for radical change; they liked their rags. And Mr. Hyatt went bankrupt trying to persuade them otherwise.
When Alfred Sloan, the man who later built GM, graduated from MIT at the head of his class in the mid-1890s, he asked his father to buy him Hyatt’s small bankrupt business. Unlike Hyatt, Sloan was willing to broaden his vision of the product. It turned out that the roller bearing was ideal for the automobile, which was just coming to market. In two years Sloan had a flourishing business; for 20 years Henry Ford was his biggest customer.
https://www.amazon.com/Innovation-Entrepreneurship-Peter-F-D...
Drucker was a critic of the idea that small firms are more innovative than large firms. He provides a wealth of innovation about the styles of innovation that large firms produce.
About this:
"But there’s a more troubling possibility. Maybe something has changed about the nature of innovation, at least in software."
When was this era? From the 1940s to the early 1980s, software development was mostly done at big firms such as IBM and Hewlett Packard. The rise of the personal computer allowed a wave of software firms, such as Microsoft and Adobe, to get going, but even then, the costs of getting such a startup going was much higher than now. The era of the really cheap software startup was basically the era of the CRUD app moved to the Web, basically 1995 to 2015, give or take a few years on either side. My point is, that era is more the exception than the rule, given the history of the industry.
It seems melodramatic to claim that a specific culture was lost, though it is absolutely true that the industry has matured and therefore the situation has changed. We are unlikely to see any more startups come out of Silicon Valley and become huge like Microsoft or Adobe or Google or Facebook.