DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for priority based on a National Stage application PCT/CN2022/116746 filed on 09/02/2022. It is noted, however, that applicant has not filed a copy of the application and the application could not be located in order to consider the priority date. The applicant should provide a copy of the PCT application in order to be granted the earlier priority date in the instant application.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's RCE submission filed on 05/11/2026 has been entered.
Response to Amendment
The amendment filed 05/11/2026 has been entered. Applicant’s amendments to the specification and claims have overcome each and every objection previously set forth in the Final Office Action mailed 02/09/2026. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant's arguments, see pg. 1-3 of the remarks filed 05/11/2026, with respect to claims 1-20 have been considered but are moot because the new ground of rejection relies on a new combination of references.
However, the new combination of references still relies on argued aspects of Zhang’s invention. On pg. 2 of the remarks, applicant argues that Zhang does not describe an “electronic messaging session” with a “topic”. Examiner respectfully disagrees for the reasons below.
Zhang states in paragraph 153 that the device for generating thumbnails based on information streams may be a messaging device. Further, paragraph 156 of Zhang states that messages may be stored on the device to support operations/applications. Thus, the method may be performed on a messaging device running messaging applications. Zhang describes that the method is applied to an information flow/stream of the messaging device (para 84-85, 153). A flow of information on a messaging device may be considered to be an electronic messaging session. Steps regarding the processing of specific messages are taught by other references since Zhang fails to further detail the structure of the messaging session beyond the citations provided above.
Further, Zhang teaches wherein the “text semantics of the text” are determined (para 84-85 and 77) and matched with image semantics (para 77). Text semantics includes an understanding of the text based on how words, phrases, etc. are combined to convey meaning. The “topic” limitation is not recited with further modifying detail that constricts what may be considered a topic of text. The determined semantics of a collection of text may be considered a “topic”. Additionally, identifying a meaning, or topic, of the text is required to carry out the intended benefit of Zhang’s invention. See paragraph 4-5 of Zhang: “When the method of generating thumbnails in the existing technology is used to display the information flow, it is easy for the thumbnails to not match the text; and the method of generating thumbnails in the existing technology uses a unified thumbnail method for all images, and the visual quality of the generated thumbnails is low, resulting in the displayed information flow being insufficiently attractive to users, thereby seriously affecting the user's experience of browsing the information flow. In view of this, there is an urgent need to provide a method and related device for generating thumbnails based on information flow, so that the thumbnails generated can be displayed when displaying the information flow, which can improve the matching between the thumbnails and the text, increase the attractiveness to users, and thus improve the user's experience of browsing the information flow.” These elements are further supported in light of the newly entered prior art, as described in the new grounds of rejection below. In view of the foregoing, applicant’s remarks are not persuasive.
Claim Objections
Claims 12-17 are objected to because of the following informalities: “wherein the computer code” should read “wherein the computer program code”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 7, 9-12, 14, 16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (CN Patent No. 114220115 A), hereinafter Zhang, in view of Back et al. (U.S. Patent No. 2022/0231979 A1), hereinafter Back, in further view of Dai et al. (CN Patent No. 107306219 A), hereinafter Dai.
Regarding claim 1, Zhang teaches a method comprising: determining, by a computing device (Zhang, para 155: “processor 1102”), a topic of an electronic messaging session (Zhang, text semantics of an information stream on a messaging device, para 84-85: “Perform natural language processing on the text in the information flow to obtain the text semantics of the text… the information flow includes text and its corresponding multiple images”; implemented on a messaging device, para 153: “device 1100 for generating thumbnails based on information streams according to an exemplary embodiment. For example, the apparatus 1100 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device”); and
by the computing device: processing the image to detect a plurality of objects in the image (Zhang, identification of key elements, see objects/elements in FIG. 3 and para 44-45 cited below);
identifying at least one first object, of the plurality of objects in the image, that are associated with the topic of the messaging session (Zhang, para 77: “the image semantics of each candidate image are matched with the text semantics of the text”; para 88: “The image semantics of the candidate image may specifically refer to the categories of each key element in the candidate image”; performed on a target image, para 77: “the image semantics of each candidate image are matched with the text semantics of the text, and at least one candidate image is determined as the target image from the multiple candidate images; based on the category and preset display size of the target image, computer vision technology is used to process the target image to generate a thumbnail”);
determining coordinates of a portion of the image that the at least one first object is included (Zhang, see Figure 3, attached below, where the location of the key element is identified in order to create the thumbnail; para 44-45: “configured to determine a candidate region in the target image based on positions of each key element in the target image; A generating subunit is configured to perform computer vision processing on the candidate area based on the preset display size to generate the thumbnail”);
PNG
media_image1.png
342
773
media_image1.png
Greyscale
generating, based on the coordinates, a thumbnail image to include the at least one first object (Zhang, para 102: “the positions of the key elements in the target image should be considered first, and a candidate area including the key elements should be determined in the target image… the candidate area is processed using computer vision technology to generate a thumbnail”; demonstrated by Figure 3).
Zhang fails to explicitly teach 1) wherein the determining a topic is performed in real-time as messages are being transmitted in the electronic messaging session, by analyzing the most recent N messages of the electronic messaging session, where N is a positive integer; 2) wherein the image processing and subsequent thumbnail generation is in response to a determination of a sending of a message including an image within the messaging session; 3) determining coordinates of a portion of the image that define a bounding box such that: the at least one first object is included in the bounding box; and at least one second object of the plurality of objects is not included in the bounding box; generating, based on the coordinates, a thumbnail image to include the at least one first object; and 4) sending the generated thumbnail image with the message to another computing device.
However, Back similarly teaches a method for generating a thumbnail image related to message content (Back, para 87: “The thumbnail generation module 1570 may generate a thumbnail image related to the notification message related to the image content”), wherein the determining a topic is performed in real-time as messages are being transmitted in the electronic messaging session, by analyzing the most recent N messages of the electronic messaging session, where N is a positive integer (Back, “a text message” includes N=1 messages of the session, para 99: “Also, for example, when the device 1000 receives a text message together with the image content, the domain identification module 1540 may identify the domain of the image content by using the received text message”; using the received text message includes first determining a domain/topic of the text message [which also describes the image content and later informs the generation of the thumbnail through notification message generation]). Back describes performing the steps of the method as messages are received in order to efficiently generate relevant information (Back, para 82). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the relevant teachings of Back with the method of Zhang in order to generate image content based on the most recent messages sent (Back, see para 99 citation above). If the messages analyzed are too old, the thumbnail image generated may be irrelevant to the current conversation, thus rendered useless to the user. In the combination of Zhang in view of Back, a person of ordinary skill in the art would be able to apply the methods taught by Zhang to the most recently received message in the messaging device of Zhang, as similarly demonstrated by the teachings of Back.
Further, Zhang discloses a base method for applying image/text processing steps to an information stream, but does not specify specific methods for performing the steps in real-time to specific messages. Back teaches a method for processing text in electronic messages in real-time as they are received. Back teaches a known technique of performing image/text processing steps on messages as a device receives them. A person having ordinary skill in the art, before the effective filing date of the claimed invention, could have applied the known technique, as taught by Back, in the same way to the method of Zhang and achieved predictable results of generating image thumbnails based on the most recent messages sent.
Additionally, when processing the image to generate a thumbnail image, Back teaches the following steps: determining coordinates of a portion of the image that define a bounding box such that: the at least one first object is included in the bounding box; and at least one second object of the plurality of objects is not included in the bounding box; generating, based on the coordinates, a thumbnail image to include the at least one first object (Back, mother and TV objects are included in the bounding box 125 and the image is cropped to thumbnail 126 and does not include a second object, such as the window curtain; para 144: “the device 100 may obtain a thumbnail image 126 related to the notification message by cropping the target image 120 to obtain a partial image in which a region 125 including the region 123 corresponding to “mother” and the region 124 corresponding to “television” is photographed”; see Figure 12, attached below). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the object-guided cropping method taught by Back with the method of Zhang in order to ensure that objects in the thumbnail image are only included if they are relevant to the message topic (Back, para 144: “For example, the device 1000 may identify “mother” and “television”, which are nouns indicating objects in the notification message 122, identify the objects corresponding to “mother” and “television” from the target image 120, and crop the target image 120 to obtain a partial image in which a region including the identified objects is photographed”).
PNG
media_image2.png
423
744
media_image2.png
Greyscale
Lastly, Dai teaches a method for sharing images between two computing devices in a thumbnail format (Dai, para 22: “wherein the picture selection module is used to select a picture selected by a user as a picture to be sent, the thumbnail generation module generates a thumbnail of the picture to be sent”; para 24: “The message server is used to forward message messages between the first mobile terminal and the second mobile terminal”), wherein responsive to a determination of a sending of a message including an image within the messaging session, performing subsequent thumbnail generation (Dai, para 44: “Step S2: The first mobile terminal generates a thumbnail based on the picture to be sent”); and sending the generated thumbnail image with the message to another computing device (Dai, para 50: “the first mobile terminal generates a message and sends it to the message server”; para 52-53: “After receiving the message, the second mobile terminal downloads the picture and its thumbnail… The second mobile terminal downloads the thumbnail image first. The original image will be downloaded only when the user clicks to view the image further”).
Zhang teaches a method regarding the generation of thumbnail images using a messaging device, but fails to explicitly teach the steps of sending the thumbnail image. Dai teaches the aforementioned method for sharing images between two computing devices in a thumbnail format. It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the thumbnail generation and sending in response to an image message, as taught by Dai, with the method/messaging device of Zhang in order to increase the image transmission speed between messaging devices (Dai, para 30: “Compared with the existing technology, the present invention can increase the loading speed of pictures during communication, improve user experience, avoid program crashes, and adapt to networks with lower transmission rates”). A person having ordinary skill in the art would be able to carry out the natural language processing method of Zhang in response to the sending of a message with an image in the messaging device of Zhang, similar to the method taught by Dai. Additionally, similar to the combination of Zhang in view of Back, Dai also teaches wherein the method is performed in real-time as messages are being transmitted in the electronic messaging system (Thumbnail is generated right before the image is sent), and could be implemented in the combined method.
Regarding claim 2 (dependent on claim 1), Zhang in view of Back and Dai teaches wherein the at least one first object is centered in the thumbnail image (Zhang, see Figure 3; para 106: “as shown in Figure 3, a schematic diagram of generating a thumbnail when the candidate area in a target image of a general category meets the preset display size and each key element in the candidate area is complete”; further supported by the combined elements of Back, demonstrated by attached figure 12).
Regarding claim 3 (dependent on claim 1), Zhang in view of Back and Dai teaches wherein the determining the topic of the electronic messaging session includes applying natural language processing (NLP) to one or more messages in the messaging session (Zhang, para 86: “natural language processing may be, for example, a TextRank algorithm or a Lexrank algorithm, etc. The TextRank algorithm or the Lexrank algorithm may automatically calculate the weight of each word in the text to extract keywords in the text as the text semantics of the text”).
Regarding claim 5 (dependent on claim 1), Zhang in view of Back and Dai teaches wherein the processing the image to detect the plurality of objects in the image comprises processing the image using a machine learning-based object detection technique (Back, object recognition model 1552; para 69: “The object recognition module 1520 may obtain identification information of one or more objects in the image content by applying a target image to an object recognition model 1552. The object recognition model 1552 may be an artificial intelligence model trained to identify an object in an image”; see also para 184 regarding artificial intelligence models). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have utilized the machine learning-based technique of Back in the invention taught by Zhang in view of Back and Dai in order to improve the object detection results by utilizing specifically trained computer vision models (Back, see para 69 citation above).
Regarding claim 7 (dependent on claim 1), Zhang in view of Back and Dai fails to teach wherein the determining the coordinates of the portion of the image includes applying image labeling to the image (Back, para 70: “For example, the object recognition module 1520 may match the identification value of the object with the visual feature of the object by labeling the identification value of the object with the visual feature of the object”).
Regarding claim 9 (dependent on claim 1), Zhang in view of Back and Dai teaches wherein the computing device is a client (Dai, first mobile terminal is a client with regard to a file server, para 14: “After obtaining the URL of the picture to be sent and its thumbnail on the file server, the first mobile terminal generates a message”) and the another computing device is a server (Dai, para 50: “message server”; see also para 52-53 with regard to information sent to the second mobile terminal).
Regarding claim 10 (dependent on claim 1), Zhang in view of Back and Dai teaches wherein the computing device is a server (Dai, first mobile terminal performs the operations to generate the thumbnail, storing data and sending it to the message server, see para 50) and the another computing device is a client (Dai, message server receives the image from the first mobile terminal).
Regarding claim 11, Zhang teaches a computing device comprising: a processor (Zhang, para 155: “processor 1102”); and a non-volatile memory storing computer program code that when executed on the processor causes the processor to execute a process (Zhang, para 164: “a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1104 including instructions, which can be executed by the processor 1120 of the device 1100 to perform the above method. For example, the non-transitory computer-readable storage medium may be a ROM”). All further claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 1 are the same as those executed in claim 11.
Regarding claim 12, all claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 3 are the same as those executed in claim 12.
Regarding claim 14, all claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 5 are the same as those executed in claim 14.
Regarding claim 16, all claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 7 are the same as those executed in claim 16.
Regarding claim 18, Zhang teaches a non-transitory machine-readable medium encoding instructions that when executed by one or more processors cause a process to be carried out (Zhang, para 59: “machine-readable medium having instructions stored thereon, which, when executed by one or more processors, enables the device to execute the method for generating thumbnails based on information flow as described in any one of the first aspects above”). All further claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 1 are the same as those carried out in claim 18.
Regarding claim 19 (dependent on claim 18), Zhang in view of Back and Dai teaches wherein the determining the topic of the electronic messaging session includes one of applying natural language processing (NLP) to one or more messages in the messaging session (Zhang, para 86: “natural language processing may be, for example, a TextRank algorithm or a Lexrank algorithm, etc. The TextRank algorithm or the Lexrank algorithm may automatically calculate the weight of each word in the text to extract keywords in the text as the text semantics of the text”) or applying a machine learning (ML) model to one or more messages in the messaging session.
Regarding claim 20, all claim limitations are met and rendered obvious by Zhang in view of Back and Dai because the method steps of claim 5 are the same as those carried out in claim 20.
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Back, Dai, and Tian et al. (Tian, Y., Wang, W., Wang, X., Rao, J., Chen, C., & Ma, J. (2010, October). Topic detection and organization of mobile text messages. In Proceedings of the 19th ACM international conference on Information and knowledge management (pp. 1877-1880).), hereinafter Tian.
Regarding claim 4 (dependent on claim 1), Zhang in view of Back and Dai fails to teach wherein the determining the topic of the electronic messaging session includes applying a machine learning (ML) model to one or more messages in the messaging session.
However, Tian teaches a method wherein determining the topic of an electronic messaging session includes applying a machine learning (ML) model to one or more messages in the messaging session (Tian, using Latent Dirichlet Allocation, pg. 1879, section 3: “we first trained a topic model with Latent Dirichlet Allocation (LDA) to measure the semantic similarity between adjacent candidate conversations, and then combining with temporal similarity, we constructed a compositive relevancy vector to process the final candidate conversation consolidation”). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the machine learning model of Tian with the method of Zhang in view of Back and Dai in order to improve the topic detection using a trained probabilistic model (Tian, pg. 1879, section 3.1: “LDA is a generative probabilistic model which is based on the hypothesis that a document can be represented as a mixture of different topics, each of which is also a probability distribution over words”). Doing so may improve topic detection for shorter samples over other natural language processing techniques (Tian, pg. 1877, section 1: “Hence, traditional approaches for TDT will not work well when applied to text messages. Several research works [3] [4] also focus on identifying the events hidden in personal and social stream objects, such as digital photo collections and social media sites contents (e.g., Flickr, YouTube, and Facebook). For its shortness and sparseness, these methods are not suitable for text messages either”).
Regarding claim 13, all claim limitations are met and rendered obvious by Zhang in view of Back, Dai, and Tian because the method steps of claim 4 are the same as those executed in claim 13.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Back, Dai, and Kee et al. (U.S. Patent No. 2019/0028605 A1), hereinafter Kee.
Regarding claim 6 (dependent on claim 1), Zhang in view of Back and Dai fails to teach wherein the determining the coordinates of the portion of the image includes applying optical character recognition (OCR) to the image. However, Kee teaches a similar system (Kee, abstract: “During operation an image will be analyzed to determine any annotation existing within the image. When annotation exists, the annotated portion is cropped and displayed as the preview (thumbnail) within, for example, a messaging application”), wherein the determining coordinates of the portion of the image includes applying optical character recognition (OCR) to the image (Kee, para 20-21: “There are multiple ways that annotation may be detected within an image. The following are some examples that are not meant to limit the broader invention. Optical Character Recognition—OCR (optical character recognition) is the recognition of printed or written text characters by a processor 303”).
Kee utilizes OCR to determine the portion of the image that will be centered in the thumbnail (Kee, see Figure 2 and para 27). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have utilized OCR, in the same way as taught by Kee, with the method of Zhang in view of Back and Dai in order to detect salient regions in the image that contain important text to be included in the thumbnail (Kee, para 10: “More specifically, in FIG. 2, a user has created an image to say “thanks” to team “hackers”. As part of this image, the user has annotated the image with a time and date of a party. This is illustrated in FIG. 2 as annotation 101. Once the image has been sent in a text, the texting application crops the portion of the image containing the text in order to display within the text message. This is illustrated in FIG. 2 as image 201. As is evident, the information about the time and date of the party is still maintained in the cropped image.”).
Regarding claim 15, all claim limitations are met and rendered obvious by Zhang in view of Back, Dai, and Kee because the method steps of claim 6 are the same as those executed in claim 15.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Back, Dai, and Barrus et al. (U.S. Patent No. 6,693,652 B1), hereinafter Barrus.
Regarding claim 8 (dependent on claim 1), Zhang in view of Back and Dai teaches generating the thumbnail image to include the at least one first object (see claim 1 rejection), but fails to teach wherein the generating the thumbnail image includes modifying an existing thumbnail image (emphasis added). However, Barrus teaches a method for generating thumbnail images (Barrus, abstract: “A multimedia message system automatically generates visual representations (thumbnails) of message or media objects”), wherein the generating the thumbnail image includes modifying an existing thumbnail image (Barrus, col 19, ln 12-29: “The dynamic updating module 818 controls the updating of any thumbnails automatically upon modification of an existing message by any user… the dynamic updating module 818 determines other instances where the object or message is displayed as a thumbnail image, and then creates a new thumbnail image and updates all objects that have an outdated image”). It would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the method of modifying an existing thumbnail image, taught by Barrus, with the method of Zhang in view of Back and Dai in order to ensure that the thumbnail image accurately represents the most recent relevant topic in the messaging session (Barrus, col 19, ln 29-33: “In this manner, the present invention ensures that the thumbnail images are an accurate reflection of the current state of a message or object and provide an invaluable source of information to the users of the system.”).
Regarding claim 17, all claim limitations are met and rendered obvious by Zhang in view of Back, Dai, and Barrus because the method steps of claim 8 are the same as those executed in claim 17.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMA E DRYDEN whose telephone number is (571)272-1179. The examiner can normally be reached M-F 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW BEE can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMMA E DRYDEN/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights