Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramid

Understand

Recently, scaling images to high resolution has received much attention in multimodal large language models (MLLMs).

• Most existing practices adopt a sliding-window-style cropping strategy to adapt to resolution increase.
• Such a cropping strategy, however, can easily cut off objects and connected regions, which introduces semantic discontinuity and therefore impedes MLLMs from recognizing small or irregularly shaped objects or text, leading to a phenomenon we call the semantic sawtooth effect.
• This effect is particularly evident in lightweight MLLMs.