Sequential Decision-Making Problems/Processes (SDPs) frequently appear in diverse real-world applications and are increasingly gaining attentions. Reinforcement Learning (RL) is successfully adopted to solve SDPs, but its effectiveness is very limited. In view of the challenge, this proposed PhD research primarily focuses on developing effective RL approaches. Among all existing RL methodologies, Policy Direct Search (PDS) algorithms have clear advantages over other approaches thanks to its ability of overcoming several key difficulties, such as âthe curse of dimensionalityâ, incompetence of solving continuous problems and lack of strong convergence guarantees. However, existing PDS algorithms have NOT (1) effectively utilized useful historical informative gradients, (2) provided suitable mechanisms to learn proper model representations, and (3) made better use of an environment model. To address these major issues, the overall goal of the thesis is to establish effective policy direct search algorithms. We aim to achieve this goal by (1) using primal-dual approximation technique to improve utilization of historical informative gradients, (2) using Evolutionary Computation (EC) techniques to learn proper model representations, and (3) using primal-dual approximation, âTeacher-inâ and Kalman-filter techniques to improve the utility of an environment model.