Funny, the BAR is also referred to as being beloved in many memoirs.
It shows up much more commonly being denigrated, though, for being utterly underwhelming vs. the MG34 / MG42- heavy, bulky, low ammo capacity and prone to jam at the damndest times. I'm not at all surprised that some subjective experiences were great, some poor; it's like any gun in that sense. I don't know why you're hot and bothered about it, though; the general opinion about the BAR is quite well supported by historians, and as the weapon's no longer issue in any army that matters, it's moot to boot.
I can see bad doctrine preventing use of an effective weapon for a generation or so, especially a generation without major wars, but not for a century, particularly not as violent a century as the late 18th/early 19th.
I would have to say that while I certainly don't see the armies of the period as completely self-defeating or unable to innovate, that this is a very mixed bag; there were a lot of mechanical concepts in the field of weaponry that were invented during the early to mid 19th Century that didn't really reach fruition until the 20th, for various reasons. For example, during the American Civil War, both Gatling and Agar's guns were bought by the Union Army, but not in numbers; part of the problem was logistical, but a lot of the problem was doctrinal; nobody had a real clue as to how these weapons were to be used yet.
It's fair to say that militaries tend to be fairly conservative about weapon systems that impact the main battle (as they conceive it- which usually lags behind the plans and capabilities of the enemy) that would change infantry doctrines; as an interesting subject, see the relatively lengthy list of projects intended to finally provide a replacement for the M16 family with something else. There have been quite a few weapons that were promising, but the ones that might have proven real game-changers, like the X23 or the G11, have faced the stiffest scrutiny. That doesn't mean that there weren't any problems with the X23, but I think that it's fair to say that they could have been sorted out, given time and funding. I think it's really interesting that when something like that is proposed that is an obvious game-changer, instead of putting a lot of money into getting the design flaws worked out, the final military spending responses are typically focused instead on trying to wring another smallish improvement in the existing arms, even though it might truly be more cost-effective, in terms of blood if not treasure, to get the next generation of weapons working.
So how the **** is it that apparently troops are carrying around custom weapons when the technology to modify them is much less prevalent, in the 18/19th centuries?
I think that this is not entirely accurate. For example,
consider the vast array of weapon systems issued to snipers during the American Civil War; a lot of factors were in play there; there were snipers with optics that weren't modern, but would be recognizable today, and they can and did hit targets at ranges that are outside the ranges non-marksmen are expected to reach, even today.
In fact, I'd argue that while one might see little modification of personal arms, and agree with you on the archeological record in that respect, that it paints a picture that isn't entirely accurate, and it ignores some things.
If you take a plowboy, who may have experience shooting a smoothbore varmint gun but no experience with a military weapon, and hand him a Springfield... what's realistic to expect, in terms of modification, even if it's allowed? During this period, optics existed, as seen at the link above, but they were expensive, fragile and not something that most of the public knew about. It's a lot different than today, where practically any 12-year-old boy can tell you more than you want to know about what he thinks about red-dot sights vs. ACOG, because these things have become part of video game culture.
In short, I think that generalization on the whole topic of customization is both inaccurate and, if one is an analyst, is liable to lead one to false premises. Culture, technology, available resources and training (not just rote, but understanding of how a weapon system works, and thus how to improve performance) have had a considerable impact on the adoption and modification of arms throughout history. The whole concept of a force armed and equipped identically is far more of a historical aberration than anything that happened during the American Civil War, frankly.
