Linux kernel has a feature or application called OOM Killer. It kills the process whenever it is out of memory. OOM decides which process to kill based on a score that it calculates which is called oom_score. You can find oom_score for each process in /proc/PID/oom_score

That rings the bell but not sure on what basis it gives a score to each process, how does it know which process have less/more ??

Well, It assigns metrics based on process like whether it is a system process, daemon process, user spawned process, and prioritizes them, checks for child process that it spawned, and their impact. Suppose if the process was started by privileged users then it has higher OOM than process spawned by Non-privilege users. Mathematically in the Linux kernel website, it writes it is calculated as (10 X percentage of memory used) i.e if a process is using 100% memory then its OOM score is 10 X 100% = 1000. But other factors like process nice value, process type might comes into play as well.

According to linux kernel.org “OOM_killer selects a process that is using a large amount of memory but is not that long lived. Processes which have been running a long time are unlikely to be the cause of memory shortage so this calculation is likely to select a process that uses a lot of memory but has not been running long. If the process is a root process or has CAP_SYS_ADMIN capabilities, the points are divided by four as it is assumed that root privilege processes are well behaved. Similarly, if it has CAP_SYS_RAWIO capabilities (access to raw devices) privileges, the points are further divided by 4 as it is undesirable to kill a process that has direct access to hardware.”