We describe the classical $o(3,2)$ $r$-matrices as generating the quantum deformations of either D=3 conformal algebra with mass-like deformation parameters or D=4 $AdS$ algebra with dimensionless deformation parameters. We describe the quantization of classical $o(3,2)$ $r$-matrices via Drinfeld twist method which locates the deformation in the coalgebra sector. Further we obtain the quantum $o(3,2)$ algebra in a convenient Hopf algebra form by considering suitable deformation maps from classical to deformed $o(3,2)$ algebra basis. It appears that if we pass from $\kappa$-deformed D =3 conformal algebra basis to the deformed D=4 $AdS$ generators basis the role of dimensionfull parameter is taken over by the $AdS$ radius $R$. We provide also the bilinear $o(3,2)$ Casimir which we express using the deformed D=3 conformal basis.