In Unified pricing you can define the base price both for sales, inventory and for purchase. If you choose to use purchase price as the base price, you will probably activate a base purchase price like this:

For the system to use this purchase price in your purchase orders you need to specify this on the released product:

This will ensure that when you create a purchase order line, then it will fetch the purchase price from the base active purchase price:

If you still see that the purchase price is fetched from trade agreement, it means that you also need to disable the trade agreement activations:

This can also applied to sales orders. So disable trade agreements you don’t need when using Unified pricing.
Another benefit you get from this is better performance, as the searching for price have several loops, and by disabling the search on different trade agreements you also get a better performance.
The following documentation is important when implementing Unified Pricing Management : https://learn.microsoft.com/en-us/dynamics365/supply-chain/unified-pricing-management/upm-base-price-determination-rules
Thanks to Vegard for figuring out this detail.