Temporal knowledge prediction is a crucial task for early event warning, which has gained increasing attention recently. It aims to predict future facts based on relevant historical facts using temporal knowledge graphs. There are two main difficulties associated with the prediction task: from the perspective of historical facts, modeling the evolutionary patterns of facts to accurately predict the query and from the query perspective, handling the two cases where the query contains seen and unseen entities in a unified framework. Driven by these two problems, we propose a novel adaptive pseudo-Siamese policy network for temporal knowledge prediction based on reinforcement learning. Specifically, we design the policy network in our model as a pseudo-Siamese network consisting of two sub-policy networks. In the sub-policy network I, the agent searches for the answer to the query along the entity-relation paths to capture static evolutionary patterns. In sub-policy network II, the agent searches for the answer to the query along relation-time paths to deal with unseen entities. Moreover, we develop a temporal relation encoder to capture the temporal evolutionary patterns. Finally, we design a gating mechanism to adaptively integrate the results of the two sub-policy networks to help the agent focus on the destination answer. To assess the performance of our model, we conduct link prediction on four benchmark datasets, and extensive experimental results demonstrate that our method achieves considerable performance compared with existing methods.