In this post I'll talk about one side effect, which is allergic reaction, now you should know that penicillin can cause type I, type II, and even type III allergic reaction, to differentiate between the three, you should know that type I is "immediate reaction" and it's caused after the body metabolizes penicillin and produce penicilloic acid.  

In some people penicilloic acid binds to the human proteins, now this combination of the penicilloic acid with the human proteins makes the immune system produce antibodies against it and produce IgE. This IgE will activate the mast cells and basophils, and the mast cell will start secreting prostaglandin and leukotrienes, and the granules will release histamine and proteolytic enzymes. On prolonged activation they will start releasing IL-1 and TNF which will cause the skin rashes, bronchoconstriction and the other signs and symptoms of allergic reaction, but this happens after minutes to penicillin administration, and in some RARE cases they may cause anaphylactic shock.
Now if type II allergic reaction develops, the immune system is activated against antigens which are present intrinsically of cell membrane and other tissues, for example when the penicillin metabolites are attached to the protein receptors of the RBC membrane they change it and the immune system will no longer recognize it, so it will develop antibodies against it, then it will release IgG, and these IgG will attach themselves to the RBC and by the aid of the complement system or the macrophages they will destroy the RBC, that's why this condition is called penicillin associated autoimmune hemolytic anemia.

In type III, the cause is immune complexes, when the penicillin and it's metabolites and circulating the body, the immune system may release IgG or IgM against them, that way it will cause "immune complexes", these complexes my be deposited in the blood vessels causing vasculitis, or in the pericardium causing pericarditis, or glomerular structure causing glomerularnephtis, or synovial membrane causing polyartharitis, but this may take long time to show, like up to 12 days.