With the oil pressure starting high and then dwindling down low would make me start looking at the bypass valve in the oil pump or the pickup tube at the base or at the screen because if it were a galley plug (and I ain't sayin that it ain't) then you should have low pressure right off the get go. There could also be some crud in a galley somewhere but I would certainly look at the oil and filter and even pull a rod and main cap off because the engine could also be eating a bearing. All bearings are technically responsible for oil pressure including the cam bearings. You can have a destroyed cam but with good journals and you'll have oil pressure. Hydraulic lifters are also responsible for oil pressure. Are they not pressurising or holding pressure? Did they move up and down and spin freely by hand in their lifter bores? Did you check this upon assembly?