Msg 4436, Level 16, State 12, Line 1
UNION ALL view 'PartWindow.dbo.vwPartitionedView' is not updatable because a partitioning column was not found.
I spent quite a few hours staring at my SQL statements and re-reading documentation before the solution hit me.
If you haven't had a chance to read the post, please do before you skip down to the answer. It was, unfortunately, a total "duh!" moment. But at the same time quite easy to overlook.