DETAILED ACTION
This action is in response to communications: Amendment filed July 27, 2026.
Claims 1-11 are pending in this case. Claim 11 has been newly amended. No claims have been newly added or cancelled. This action is made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, and 6-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanson et al. (US 2011/0234613).
As to claim 1, Hanson et al. disclose a text information generation device (Figure 2, computer system 105) that generates text information (e.g. text) based on a background image (e.g. a presentation layout including a background, e.g. displayed as a digital page(s) in a digital spread) and one or more images (e.g. one or more digital images 220), the text information generation device (e.g. computer system 105) comprising: a processor, wherein the processor (e.g. data processing apparatus 215) is configured to execute: first acquisition processing of acquiring first information (e.g. acquiring cluster of images 255 and/or significance measures 260 derived from features 225 of digital images 220) related to at least one of a layout of the image in the background image or the background image (e.g. related to at least one of a layout of one or more digital images in one or more background images)([0053] notes data processing apparatus 215 can dynamically generate a layout for multiple digital images and place the multiple images in the layout based on significance measures of the multiple digital images, where a digital image with a higher significance measure can be displayed more prominently in the layout and additional digital images can be displayed by descending significance measures, [0056] further notes data processing apparatus 215 creates layouts for digital media presentations using both clusters 255 and the significance measures 260, [0067] notes a presentation layout is a background against which one or more multiple digital images are presented, a presentation layout can include themes with accompanying related text and images, e.g. birthday theme can include images of balloons that will be displayed along with the digital images); second acquisition processing of acquiring second information related to the image (e.g. acquiring features 225 related to one or more digital images 220)([0030], [0031], [0033] notes digital images are associated with features, e.g. features 225, that describe the image, such features include image metadata that describes properties of an image, e.g. a time of capture, a geographic location of capture, a description associated with the image either by the device using which the image was captured or by a user or both); and generation processing of generating the text information based on the first information and the second information (e.g. [0084] notes computer system 105 can automatically display text on the digital page spread 400 that describes digital images in the digital page spread 400, the text can be determined based on metadata associated with the digital images in the digital page spread 400, thus based on the features of the digital images, e.g. “second information,” where [0088] further notes the text, e.g. caption, can be displayed as an overlay on top of the bounded region 420, or in a region of the left digital page 405 outside of the bounded region 420, e.g. above the bounded region 420, thus based on the cluster of images and/or significance measures which defines the presentation layout, e.g. “first information”).
As noted above, Hanson et al. disclose automatically generating text based on features of digital images, e.g. considered the “second information” and further notes the text may be placed in a particular area/region based on the presentation layout (background), e.g. caption/text placed above a larger photograph in a digital page, e.g. which has been determined based on the cluster of images and significance measures described. Therefore, it is obvious that the text is further generated and placed based on the cluster of images and significance measures, e.g. considered the “first information,” yielding predictable results, without changing the scope of the invention.
As to claim 4, Hanson et al. disclose wherein the first information related to the layout is information related to a blank region in which the image is not disposed in the background image (e.g. as noted in claim 1, a presentation layout (background) may be displayed as a digital page(s) in a digital spread, where Figure 3, [0071] notes example of digital page 300 including multiple bounded regions, where each bounded region may be used to present each digital image in a cluster, the size, shape, placement, and orientation of the bounded regions can be variable and determined based on the significance measure of the digital images presented in the bounded regions, thus it may be considered blank regions are outside of the bounded regions, “…in which the digital image is not disposed in the background image…”).
As to claim 6, Hanson et al. disclose wherein the processor is configured to generate, in the generation processing, the text information based on the first information related to the layout (e.g. cluster of images 255 and/or significance measures 260), a rule for generating the text information according to the layout (e.g. display text as an overlay on top of a bounded region, or outside of the bounded region, e.g. above the bounded region, or display significant text as a caption and not displaying insignificant text), and the second information (e.g. features of one or more digital images)(e.g. as noted in claim 1, text may be generated based on features of one or more digital images, e.g. metadata, and displayed in a particular area, e.g. on top of a bounded region or above a bounded region of a larger image of the cluster of images, thus based on the cluster of images and/or significance measures, such that the images are not obstructed, further considered a “rule”).
As to claim 7, Hanson et al. disclose wherein, in a case where the layout is a layout in which two or more images of different sizes are disposed on the background image (e.g. as noted in claim 1, a presentation layout (background) may be displayed as a digital page(s) in a digital spread, where Figure 4, [0078], [0079] notes digital page spread 400 with left digital page 405 and right digital page 410, where the layout of the left digital page 405 includes a single bounded region 420, and the layout of the right digital rage 410 includes multiple bounded regions 425, 430, 435, 440, 445, and 450, with single bounded region 420 illustrated larger than any other bounded region, where [0071] notes each bounded region may be used to present a digital image of a cluster), the processor is configured to generate, in the generation processing, the text information based on at least the second information related to a largest image ([0080], [0081] notes determining a layout for the digital images in a cluster based on significance measures 260, e.g. the computer system 105 determines that a digital image has a highest significance measure in the cluster and displays the first digital image in bounded region 420, then computer system 105 determines that a second digital image has a second higher significance measure and displays the second digital image in bounded region 425, and display the remaining digital images in the cluster in bounded regions 430, 435, 440, 445, and 450 by descending significance measure, [0084] notes generating text to be displayed on digital page spread 400, [0088] further notes displaying text, e.g. caption “European Vacation” as an overlay on top of the bounded region 420 or in a region of the left digital page 405 outside of bounded region 420, e.g. above bounded region 420, thus related to the largest image) .
As to claim 8, Hanson et al. disclose wherein, in a case where the layout is a layout in which two or more images of the same size are disposed on the background image (e.g. as noted in claim 1, a presentation layout (background) may be displayed as a digital page(s) in a digital spread, where Figure 5 illustrates digital page spread 510 having bounded regions of the same size, where [0098] notes digital page spread 510 having digital images with highest significance measures), the processor is configured to generate, in the generation processing, the text information based on the second information related to each of the two or more images ([0093] notes text can be displayed as a caption, a title, or a summary of one or more digital images, where it may be considered the text information relates to both of the images when displayed equal sizes).
As to claim 9, Hanson et al. disclose wherein the processor is configured to: execute processing of generating an image attached with text information in which the text information is disposed in at least any one of the background image or the image ([0033], [0093] notes features, e.g. features 225, can additionally include text associated with a digital image, the text can be received from a user managing the digital image and can be, for example, a data file name under which the user stores the image, a caption, such as text, that the user associates with the image, and the like); and further execute processing of deciding a disposition method of a text indicated by the text information in the image attached with text information based on at least one of the first information or the second information ([0084] notes computer system 105 can automatically display text on the digital page spread 400 that describes digital images in the digital page spread 400, the text can be determined based on metadata associated with the digital images in the digital page spread 400, thus based on the features of the digital images, e.g. “second information”).
As to claim 10, Hanson et al. disclose a text information generation method of generating text information (e.g. text) based on a background image (e.g. a presentation layout including a background, e.g. displayed as a digital page(s) in a digital spread) and one or more images (e.g. one or more digital images 220) by a processor (data processing apparatus 215 of computer system 105 of Figure 2), the text information generation method comprising: by the processor (data processing apparatus 215), performing the steps as performed by the processor of the text generation device of claim 1. Please see the rejection and rationale of claim 1.
As to claim 11, Hanson et al. disclose a non-transitory computer-readable recording medium (Figure 2, computer storage medium, e.g. data storage 210) on which a program for causing a computer (data processing apparatus 215) to execute each of the steps included in the text information generation method ([0110] thru [0114] notes computer storage medium for storing computer programs for execution by, or to control the operation of, data processing apparatus) according to claim 10 is recorded. Claim 10 is similar in scope to claim 1. Please see the rejection and rationale of claim 1.
Claim(s) 2, 3, and 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanson et al. (US 2011/0234613) as applied to claims 1 and 4 above, and further in view of Kokemohr et al. (US 10,049,477).
As to claim 2, Hanson et al. do not disclose, but Kokemohr et al. disclose wherein the processor is configured to execute, in the generation processing, output processing of setting a style based on at least any one of the first information or the second information, and outputting the text information according to the style (Figure 3, step 306, column 16, lines 10 thru column 17, lines 31 notes determining one or more text attributes of the generated text, such as font, color, size, layout, letter case, and/or other text attribute types, which may include automatically selecting text attributes that are predefined to be associated with particular text, with determined image characteristics of the image, and/or with a particular visual style determined for the image, e.g. older-style cursive or script font can be a text attribute associated with a particular recognized types of image content, while a lively or chaotic font can be associated with image content including lots of faces, persons, or a city landscape, where characteristics of the image may be considered at least “second information”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Hanson et al.’s system and method of generating text information with Kokemohr et al.’s method of generating text information including setting a style of the text information such that the style, e.g. font, of the text can be associated with a particular visual style of the image, thus enhancing techniques of automatically generating text without user intervention (see at least column 16, lines 10 thru column 17, lines 31 of Kokemohr et al.).
As to claim 3, Hanson et al. modified with Kokemohr et al. disclose wherein the first information related to the background image is information related to a color pattern of the background image (Hanson, e.g. as noted in claim 1, presentation layout (background) can include themes related to the digital images, where [0085], [0086] further notes determining a prominent color in the digital images in the digital page spread and further determining a complementary color to the prominent color and display the complimentary color as a background color on the digital page spread).
As to claim 5, Hanson et al. do not disclose, but Kokemohr et al. disclose wherein the processor is configured to decide, in the generation processing, at least one of the number of characters of a text included in the text information or a display size of the text based on a size of the blank region (Kokemohr, step 306, column 16, lines 10-18 notes determining one or more text attributes of the generated text, such as font, color, size, layout, letter case, and/or other text attribute types, where column 17, lines 65 thru column 18, lines 41 notes the location in the image at which the generated text is displayed, e.g. positioned in the image to minimize obscuration of content subjects of the image, as candidate areas that have small levels of color noise, where larger candidate areas can allow the generated text to have a larger size while smaller candidate areas may require the generated text to be of a smaller size, where column 18, lines 59 thru column 19, lines 8 notes generating a text area, such as a text box, in which the generated text is to be placed, the generated text area can be sized and shaped to fit the generated text, where unobstructed regions of images and/or a text area may be considered “blank region”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Hanson et al.’s system and method of generating text information with Kokemohr et al.’s method of generating text information based on a number of characters or display size of the text such that text generated may not obstruct significant features of the image(s), thus enhancing techniques of automatically generating text without user intervention (see at least column 17, lines 65 thru column 19, lines 8 of Kokemohr et al.).
Response to Arguments
Applicant’s arguments, see page 5, filed July 27, 2026, with respect to claim 11 have been fully considered and are persuasive. Applicant amends claim 11 to recite, “A non-transitory computer-readable recording medium…” Therefore, the 35 U.S.C. 101 claim rejection of claim 11 has been withdrawn.
Applicant’s arguments filed July 27, 2026 have been fully considered but they are not persuasive. Applicant argues on pages 5-8 regarding independent claim 1 that the prior art of record, e.g. Hanson, fails to teach or suggest the limitations of independent claim 1 as recited, more specifically, “…Hanson neither discloses nor suggests the subject matter of claim 1 with respect to at least the two claim features (B) “first acquisition processing” and (D) “generation processing.”
Applicant argues “…(i) Regarding feature (B): The clusters 255 and the significance measures 260 of Hanson, which the Examiner found to correspond to the “first information” of the present application, are both information about the digital images themselves. The significance measures 260 are scores computed from image metadata (viewing counts, e-mailing, printing history, user ratings, and the like) and from image content (face recognition and the like) (Hanson, paragraphs [0046]-[0052]), and the clusters 255 are logical groupings of images based on common characteristics (id., paragraphs [0041] and [0042]). Within the framework of claim 1 of the present application, such information corresponds at most to the “second information” related to the image, and is categorically different from the “first information” related to “a layout of the image in the background image” or to “the background image” (which is defined as an image functioning as a mounting board on which the disposition images are disposed in paragraph [0019] of the present specification). Furthermore, the layout in Hanson is the output of processing that is generated based on the significance measures (id., paragraphs [0053] and [0056]); it is not information acquired as an input for text generation. Applicant respectfully submits that the direction of the data flow in Hanson is the reverse of that in the present application…” (last paragraph of page 5 continued to page 6 of the Amendment filed).
In reply, Hanson describes a system and method of automatically generating digital media presentation layouts based on image features. As noted in the rejection above regarding claim 1, Hanson discloses its data processing apparatus can dynamically generate a layout for multiple images based on significance measures of the multiple images, e.g. where the significance measures may be determined based on features of the digital images ([0053], [0055]). The layout may be a background against which one or more multiple digital images are presented, may be for a single digital image or multiple digital images, and may include themes ([0067]). This layout that is generated is considered the ”first information.” The features, considered the “second information,” may include metadata that describes properties of an image, such as time of capture, a geographic location of capture, a description associated with the image either by the device using which the image was captured or by a user or both, file size, file type, properties of the device using the which the image was captured, e.g. camera focal length, aperture settings, user ratings, a duration for which a user has viewed the digital image, a number of times a user has emailed the digital image, etc. ([0030], [0057]), Because the claim does not specifically recite or define how or when the “first information” is acquired, the generation of the layout is considered “acquiring first information.” Further, because the claim does not specifically recite or define how or when the “second information” is acquired, the features may be acquired upon receiving and analyzing the digital images ([0056]). Although the layout, e.g. “first information,” may be generated based on the significance measures, e.g. further based on the features, of the digital images, the layout is considered separate and distinct from the features themselves. Further still, Hanson discloses the significance measures may be based on a feature or features ([0055]), thus denoting not all features may be used to generate a layout, thus different features may be considered “second information,” further separate and distinct from the “first information.” Therefore, it is believed Hanson teaches “…first acquisition processing of acquiring first information related to at least one of a layout of the image in the background image or the background image…” similarly recited by independent claims 1, 10, and 11.
Applicant further argues “…(ii) Regarding feature (D): The generation processing of claim 1 requires that the generation of the content of the text information be based on both the first information and the second information. Hanson, by contrast, appears to treat the determination of the text content (which is based on image metadata; id., paragraph [0084] and the first part of paragraph [0088]) and the placement of the text (as an overlay on top of the bounded region 420 or in a region outside of it; id., the latter part of paragraph [0088]), as processes independent of each other, and nowhere describes using the size, position, or layout of the bounded regions, or the nature of the background, as a determinant of the content of the text. Applicant respectfully submits that the Examiner’s reasoning-that because text may be placed based on the layout, it would also be obvious to generate text based on the layout-is an unsupported leap from text placement to text content generation, and lacks the “articulated reasoning with some rational underpinning” required by KSR Int'l Co. V. Teleflex Inc., 550 U.S. 398, 418 (2007), and MPEP § 2142…” (first paragraph of page 6 of the Amendment filed).
In reply, claim 1 recites, “…generation processing of generating text information based on the first information and the second information.” The claim does not specifically recite or define “text information,” let alone to only include text “content.” However, as noted in the rejection above regarding claim 1, Hanson explicitly discloses automatically generating and displaying text, where that the text can be determined, e.g. generated, based on the metadata (e.g. included in features as “second information”) associated with the digital images in the digital page spread ([0084]). Figure 4 illustrates the text (e.g. text “content”) generated as “European Vacation” based on the metadata of the digital images ([0084]), where the text is further determined, e.g. generated, to be placed (e.g. text “placement”) above the left digital page 405 as this is the title and the beginning of the digital media presentation ([0088]), thus the text placed above the larger digital page 405 because of the layout, e.g. “first information.” Therefore, the “content” and “placement” of the text may both be encompassed in “text information” which is generated based on the layout, e.g. “first information,” and the features of the digital images, e.g. “second information,” thus teaching “…generating processing of generating the text information based on the first information and the second information…” similarly recited by independent claims 1, 10, and 11.
Applicant further argues “…(iii) Regarding the difference in the technical problem addressed: The problem addressed by the present application is achieving harmony between the impression given by the background image and the layout, on the one hand, and the content of the text, on the other; that is, if text is generated based only on the images, the text does not match the impression of the background image and the layout, leaving a sense of incongruity in the overall impression of the photo book (present specification, paragraphs [0004]-[0006]). Hanson's objective, in contrast, appears to automatically determine the display sizes and placements of images according to their significance so as to reduce the burden on the user (Hanson, paragraphs [0011] and [0012]). Hanson does not even recognize the problem addressed by the present application. It is well settled that the problem addressed and solved by the claimed subject matter must be given consideration in resolving the ultimate legal conclusion of obviousness under 35 U.S.C. § 103. North American Vaccine, Inc. V. American Cyanamid Co., 7 F.3d 1571, 28 USPQ2d 1333 (Fed. Cir. 1993); Northern Telecom, Inc. V. Datapoint Corp., 908 F.2d 931, 15 USPQ2d 1321 (Fed. Cir. 1990); In re Nomiya, 509 F.2d 566, 184 USPQ 607 (CCPA 1975). Further, a cited reference lacking any recognition of this problem provides no motivation to modify Hanson so as to generate the content of text based on information related to the background image or the layout. Thus, the present claimed subject matter is nonobvious…” (last paragraph of page 6 continued to page 7 of the Amendment filed).
In reply, as noted in the Examiner’s response above, the features of the digital images may include various information/characteristics/specifications about the image itself, but may also encompass other information, e.g. properties of the device using which the image(s) was captured, e.g. camera focal length, aperture settings, etc. as well as user ratings, a duration for which a user has viewed the digital image, a number of times a user has emailed the digital image, etc. ([0030], [0057]). Further noted in the Examiner’s response above, the text information may be generated based on the metadata (e.g. included in features as “second information”) associated with the digital images in the digital page spread as well as the layout generated (e.g. “first information”)([0084],[0088]). Hanson describes its system and method of automatically generating digital media presentation layouts based on image features. The method further comprises automatically generating text based on the generated layout and image features as noted in the Examiner’s response above. Although Hanson does not explicitly state its solving the same problem as the Applicant’s present invention, this does not mean that Hanson does not achieve the goal of solving the same problem.
Applicant further argues “… (iv) The separate patentability of dependent claim 4, 6-9, 10, and 11 is advocated. With respect to claim 4, the Examiner’s assertion that the blank region would be understood to lie outside the bounded regions is not a disclosure, but an unsupported inference; that the bounded regions of Hanson are merely regions for presenting images (id., paragraph [0071]); and that Hanson discloses neither “the acquisition of information related to a blank region in which the image is not disposed” nor the use of such information for text generation…” (first and second paragraphs of page 7 of the Amendment filed).
In reply, in a digital page spread, as illustrated in Figure 4, the text “European Vacation” may be displayed within bounded region 420 of the left digital page 405, where bounded region 420, although not illustrated, would include a digital image, similarly as bounded regions 430-450 of right digital page 410 ([0079]). Alternatively, Hanson further discloses the text “European Vacation” may be displayed in a region of the left digital page 405 outside of bounded region 420, e.g. above bounded region 420 ([0088]). Comparing to digital page spreads of Figure 5, the regions outside of the bounded regions, are blank (e.g. white, empty), where the regions of bounded regions 525-550 of Figure 5 illustrates each digital page spread may not be “blank” (illustrated as having patterns). Therefore, it is believed Hanson still teaches the limitations of claim 4 as recited.
Applicant further argues “…(iv) The separate patentability of dependent claim 4, 6-9, 10, and 11 is advocated…With respect to claim 6, the passages cited by the Examiner (such as the overlay display on the bounded region) all concern the placement of text and do not disclose “a rule for generating the text information according to the layout”…” (first and third paragraphs of page 7 of the Amendment filed).
In reply, as noted in the Examiner’s response above, the text “content” and “placement” considered the claimed “text information” is based on both the features of the digital images as well as the layout of the digital media presentation. Please see the Examiner’s responses regarding independent claims 1, 10, and 11 above.
Applicant further argues “…(iv) The separate patentability of dependent claim 4, 6-9, 10, and 11 is advocated…Dependent claims 4 and 6-9 are patentable at least by virtue of their dependency from claim 1, and that method claim 10 and amended recording-medium claim 11 are patentable for the same reasons as claim 1. Kokemohr does not remedy the above-argued deficiencies of Hanson. Kokemohr is cited only for the setting of text attributes such as font, color, and size (col. 16, line 10 to col. 17, line 31) and for the determination of the display size and position of text according to unoccluded candidate regions of an image (col. 17, line 65 to col. 19, line 8). Kokemohr does not disclose (i) the acquisition of the first information related to the background image or the layout, or (ii) the generation of the content of text based on both the first information and the second information. Therefore, even if Hanson and Kokemohr were combined, as argued by the Examiner, the combination would still fail to arrive at the subject matter of claim 1, from which claims 2, 3, and 5 depend (MPEP § 2143.03: all claim limitations must be taught or suggested by the prior art)…” (first and fourth through sixth paragraphs of page 7 of the Amendment filed).
In reply, Kokemohr is cited for teaching limitations of claims 2, 3, and 5 not taught by Hanson, where Hanson is used solely for teaching the limitations of independent claim 1. Based on the Examiner’s responses above, it is believed Hanson still teaches the limitations of at least independent claims 1, 10, and 11 as recited, thus does not depend on teachings of Kokemohr for independent claims 1, 10, and 11. Please see the Examiner’s responses regarding claims 1, 10, and 11 above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACINTA M CRAWFORD whose telephone number is (571)270-1539. The examiner can normally be reached 8:30a.m. to 4:30p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACINTA M CRAWFORD/Primary Examiner, Art Unit 2617
