Thanks to johnb on the Factor's discord channel I got answers to these two questions:
In terms of nicer way to do a nested map nicely, fried quotations are the answer:
'[ _ map ] map
and in order to call a quotation inside a word definition, one needs to use call( stack effect )
form:
: nested-map ( a quote -- a' ) '[ _ '[ _ call( x -- x' ) ] map ] map ;