J

Unsurprisingly this is the kind of problem that J is really good at. The dyadic case (table) of the adverb / is doing all the heavy lifting here: it makes a higher rank tensor by traversing items of the specified rank on each side and combining them according to the remaining frame of each side's shape. The hard part is arranging the arguments so that your resulting matrix has its axes in the correct order.

data_file_name =: '4.data'

NB. cutopen yields boxed lines, so unbox them and ravel items to make a letter matrix
grid =: ,. > cutopen fread data_file_name
NB. pad the grid on every side with #'XMAS' - 1 spaces
hpadded_grid =: (('   ' & ,) @: (, & '   '))"1 grid
padded_grid =: (3 1 $ ' ') , hpadded_grid , (3 1 $ ' ')
NB. traversal vectors
directions =: 8 2 $ 1 0 1 1 0 1 _1 1 _1 0 _1 _1 0 _1 1 _1
NB. rpos cpos matches rdir cdir if the string starting at rpos cpos in
NB. direction rdir cdir is the string we want
matches =: 4 : 0
*/ ,'XMAS' -: padded_grid {~ <"1 x +"1 y *"1 0 i. 4
)"1
positions =: (3 + i. 0 { $ grid) ,"0/ (3 + i. 1 { $ grid)
result1 =: +/, positions matches/ directions

NB. pairs of traversal vectors
x_directions =: 4 2 2 $ 1 1 _1 1 1 1 1 _1 _1 _1 _1 1 _1 _1 1 _1
NB. rpos cpos x_matches 2 2 $ rdir1 cdir1 rdir2 cdir2 if there is an 'A' at
NB. rpos cpos and the string in each of dir1 and dir2 centered at rpos cpos
NB. is the string we want
x_matches =: 4 : 0
NB. (2 2 $ rdir1 cdir1 rdir2 cdir2) *"1 0/ (_1 + i.3) yields a matrix
NB. 2 3 $ (_1 * dir1) , (0 * dir1) , (1 * dir1) followed by the same for dir2
*/ ,'MAS' -:"1 padded_grid {~ <"1 x +"1 y *"1 0/ _1 + i. 3
)"1 2
result2 =: +/, positions x_matches/ x_directions