With every new generation, Field Pro-grammable Gate Arrays (FPGAs) are getting more complex and so are their back end flow. Routing is an important step of FPGA back end flow that takes a lot of time. Making it more efficient in terms of execution time without the loss of quality is a huge challenge. In this work, we propose to use Reinforcement Learning(RL) based routing technique to make the FPGA routing faster. We use a comprehensive set of homogeneous and heterogeneous benchmarks to compare the RL-based technique with the conventional negotiated congestion driven routing technique. Experimental results reveal that for quick turn around, when compared to negotiated congestion technique, the RL-based technique gives, on average, 35% more accurate results about the final design. Moreover, for the complete routing step, the RL-based technique gives 30% speed up while giving similar quality of results.