DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the following communication: an amendment filed on 02/11/2026.
Claims 1-20 are currently pending and presented for examination.
Response to Arguments
Applicant’s remarks and amendments filed on 02/11/2026 with respect to claims prior art rejection have been considered but are moot because the arguments do not apply to the combination of the references being used in the current rejection. Rejection of claim 20 under 35 U.S.C. 101 is withdrawn, since the claim has been amended to overcome the rejection.
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US Pub. No.: US 2021/0272253 A1), in view of Cofer et al. (US Pub. No.: US 2002/0125435 A1).
Regarding claim 1, Lin et al. discloses an electronic device ( Par 45; the image
merging system is located on a client device that has image capturing hardware (i.e., a
camera), which captures the set of images. ) comprising:
a display ( Par a61; a client device displaying a graphical user interface 302) ; at
least one camera (Para 45; a client device that has image capturing hardware (i.e., a
camera), which captures the set of images.);
a memory ( Para 158; memory that includes a first image having faces of one or more persons, a second image having faces of one or more persons where a person in the second image is not in the first image, a face detection model, and/or a segmentation model.) having stored thereon a missing image integration module (MIIM)
for integrating a non-contemporaneously captured image of a person to a device
captured image;
and at least one processor (Para 149; the components 1210-1234 can include one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices, such as a client device (e.g., a mobile client device) or server device. When executed by the one or more processors, the computer-executable instructions of the image merging system 1204 can cause a computing device to perform the feature learning methods described herein communicatively coupled to the display, the at least one camera, and the memory, the
at least one processor executing program code of the missing image integration
module, which enables the electronic device to:
detect activation of the at least one camera (Para 46; the image merging system
communicates with a camera or a client device having a camera and detects when multiple images are being captured (e.g., at images are captured at the same location and/or within a time threshold), which triggers creation of a composite group photo.) to capture a first image within a field of view of the at least one camera;
identify at least one first individual within a first preliminary image (Para 47; the series of acts 200 includes an act 206 of the image merging system utilizing the face detection model 204 to identify faces within the images. For instance, the face detection
model 204 is a face detection neural network trained to identify and isolate faces of
persons within each of the images.);
determine if at least one second individual is missing from the first preliminary image based on a comparison of individuals identified within the first preliminary image to a first group of individuals that are normally included within captured images comprising the at least one first individual ( Para 59-63; if a family has a child at college and the child does not appear in a family photo, the image merging system can add a picture of the child (e.g., the second image) to the family photo (e.g., the base image); the woman on the left in the second image 308 is not in the base image 306.) ;
and in response to determining that the at least one second individual is missing
from the first preliminary image, present on the display a graphical user interface (GUI)
that contains a user-selectable option to enable a missing image mode to electronically integrate an image of the missing at least one second individual into the captured first image (Figs. 3B-10; Para 61- 85; Para 119-126; The interface 302 includes an image editing application 304 that provides various image editing tools and options. In addition, the image editing application 304 includes a base image 306 and a second image 308. the image merging system determines whether a face in the second image does not have a match with a face in the base image. For example, for each face detected in the second image, the image merging system determines if any face descriptors in the base image are within the similarity threshold. If not, then the face in the second image corresponds to a missing person. Otherwise, the face in the second image matches a face in the base image. If the missing person has their arms around the given person in the second image, the image merging system can remove the given person from the base image and add the segmented image if the missing person with their arms around the given person. If the physical connection is between two people missing from the base image, the image merging system can add the “couple” from the second image to the base image. The image merging system provides the merged image to a client device associated with a user. In particular, the image merging system can provide the merged image to a user in response to a user request to generate the merged image from a set of images. In some implementations, the image merging system provides the merged image within an image editing application and allows the user to further modify the merged image (e.g., the composite group photo).
However, Lin et al. does not disclose determine, in real-time during camera activation and prior to capturing a final image, compare individuals identified within the first preliminary image to a pre-established first group of individuals stored in the memory.
Cofer et al. discloses determine, in real-time during camera activation and prior to capturing a final image, compare individuals identified within the first preliminary image to a pre-established first group of individuals stored in the memory (Para 42-50; a live image of the monitored area. Step 100 compares the reference image and the live image to determine selected differences. Step 102 determines if the differences identified in step 100 exceed the threshold value specified in step 90. If the differences exceed the threshold value, control is passed to step 104. Step 104 signals that an object is present in the monitored area. If the differences do not exceed the threshold value, control is passed to step 106. Step 106 signals that an object is not present in the monitored area; wherein live image is in real-time) .
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to compare live images as disclosed in Cofer et al. in order to offer user immediate feedback and verification to produce images with desired result and improve efficiency and reduce processing time.
Regarding claim 2, Lin et al. discloses wherein the at least one processor: in response to detecting selection of the missing image mode: identifies a previous image containing the at least one second individual ( Para 63-67 ; Figs.3A-10; the woman on the left in the second image 308 is not in the base image 306. ) ; crops out an image comprising the at least one second individual from the previous image (Figs. 7B, 7C; Para 90-91; the image merging system generates an image layer that positively selects pixels associated with the missing person 712 from the second image 308 while negatively selecting other pixels within the second image 308; FIG. 7C shows the image merging system generating a segmented image 706 of the missing person 712 from the second image 308) ; integrates the cropped image into the first preliminary image to generate a first composite image including the cropped image of the at least one second individual ( Figs.8-10; Para 112-121; Upon identifying the base image, the available location, and the segmented image, the image merging system can generate a merged image. As shown, the series of acts 1000 includes an act 1008 of the image merging system generating a merged image by inserting the segmented image into the base image at the available location. ) ; and displays the first composite image on the display ( Para 131; Fig.11; a graphical user interface of a merged image in accordance with one or more implementations. As shown, the image merging system updates the graphical user interface 302 to show the merged image 1102. Indeed, the merged image 1102 is displayed within the image editing application 304 on the client device 300, as introduced previously. ) .
Regarding claim 3, Lin et al. discloses The electronic device of claim 2, wherein to crop the image from the previous image, the at least one processor:
processes the previous image through an artificial intelligence engine, which identifies the at least one second individual and a background in the previous image ( Figs. 7B, 7C; Para 50; the face detection model 204 is a neural network ; machine learning model that includes that includes interconnected artificial neurons that communicate and learn to approximate complex functions and generate outputs based on a plurality of inputs provided to the model; the image merging system generates an image layer that positively selects pixels associated with the missing person 712 from the second image 308 while negatively selecting other pixels within the second image 308. Wherein Fig 7B shows that the background was not selected ) ; and
removes the background from the previous image to generate the cropped image based on the remaining at least one second individual after the background has been removed (Fig. 7C; Para 91; shows the image merging system generating a segmented image 706 of the missing person 712 from the second image 308. For example, in one or more implementations, the image merging system utilizes the object mask 704 to select the missing person 712 from the second image 308. Then, the image merging system copies the missing person 712 into a new image—the segmented image 706 ).
Regarding claim 4, Lin et al. discloses The electronic device of claim 2, wherein the at least one processor:
identifies at least one first image characteristic of the first preliminary image (Para 124; lending the segmented image with the base image ; background colors can be matched ; therefore, the background color is determined from base image);
generates a modified cropped image by adjusting one or more second image characteristics of the cropped image to match the at least one first image characteristics ( Para 124; blending includes adjusting the contrast, shading, hue, saturation, sharpness, and/or resolution of the segmented image to match the base image); and
incorporates the modified cropped image into the first image to generate the first composite image (Para 124; the image merging system utilizes a blending model (e.g., a blending neural network) to blend the segmented image into the background image.) .
Regarding claim 5, Lin et al. discloses The electronic device of claim 2, wherein the at least one processor:
identifies a first image context of the first preliminary image based on characteristics of the first image (Para 46; the image merging system communicates with a camera or a client device having a camera and detects when multiple images are being captured (e.g., at images are captured at the same location and/or within a time threshold), which triggers creation of a composite group photo);
identifies a second image context of the previous image based on characteristics of the previous image (Para 46; the image merging system communicates with a camera or a client device having a camera and detects when multiple images are being captured (e.g., at images are captured at the same location and/or within a time threshold), which triggers creation of a composite group photo);
determines if the first image context is substantially similar to the second image context; and in response to determining that the first image context is substantially similar to the second image context, selects the previous image containing the at least one second individual for cropping ( Para 48-59; the image merging system can validate the set of images based on timestamp, such as determining that the images in the set of images are captured within a threshold period of time (e.g., 5 minutes, 1 hours, 1 day). As another example, the image merging system validates based on location, such as determining that images have geotags within a threshold distance of each other (e.g., 10 feet, 50 feet, 1 mile). As yet another example, the image merging system validates based on capture device, such as determining that the device identifier of the capturing device (e.g., camera or smartphone) matches for each of the images. Para 67-77;
Fig. 4 includes a series of acts 400 of the image merging system determining faces missing in the base image in accordance with one or more implementations. As illustrated, the series of acts 400 includes an act 402 of the image merging system identifying face descriptor for each face from each of the images. Para 86-90; Figs. 7B,7C; The segmentation model 604 can correspond to one or more deep neural networks or models that select an object based on bounding box parameters corresponding to the object within an image (i.e., the second image). ) .
Regarding claim 6, Lin et al. The electronic device of claim 2, wherein the missing at least one second individual is an electronic device user that is using the electronic device and an image of the electronic device user is cropped from the identified previous image to integrate into the first composite image ( Para 60-61; he base image and the second image each include at least one person, in one or more implementations, the base image does not include any people (or at least no faces are detectable within the first image). For example, a user hiking by herself takes a picture of a mountain landscape and later captures an image of herself. Here, the image merging system can assist the user in adding her picture to the mountain landscape using the actions and techniques described herein. ) .
Regarding claim 7, Lin et al. discloses wherein the at least one processor: presents on the display a GUI that contains a user-selectable option to store one or both of the first preliminary image and the first composite image; and stores a corresponding one or both of the first preliminary image and the first composite image based on a received selection (Fig. 11; Para 126-132; the image merging system provides a link to the merged image or a downloadable copy of the merged image to the user where the user can access, save, and/or share the merged image. the image merging system updates the graphical user interface 302 to show the merged image 1102. Indeed, the merged image 1102 is displayed within the image editing application 304 on the client device 300, as introduced previously. the image merging system displays one or more updates of the graphical user interface 302 corresponding to an intermediary action, for example, based on a user request and/or user preferences.) .
Regarding claim 8, Lin et al. discloses The electronic device of claim 1, further comprising: at least one input device ( Figs. 13,15; Para 151; client device and a service device connected via a network 1308 ) communicatively coupled to the at least one processor; and the at least one processor: in response to activation of the first camera, activates the at least one input device; receives communication input from the at least one input device (Para 46, 154-155; the image merging system communicates with a camera or a client device having a camera and detects when multiple images are being captured (e.g., at images are captured at the same location and/or within a time threshold), which triggers creation of a composite group photo. the image editing server system 1306 communicates with the image merging system 1204 on the client device 1302 to facilitate the functions, operations, and actions previously described above with respect to the image merging system 1204. For example, the image editing server system 1306 can provide digital content (e.g., a web page) to a user on the client device 1302 and facilitate generating merged images from a set of images via a network-provided graphical user interface.) ; and identifies the at least one second individual at least partially based on a context of the communication input received from the at least one input device ( Para 157-168; Fig. 14; he series of acts 1400 is implemented on one or more computing devices, such as the client device 1302, the server device 1304, or the computing device 1200. In addition, in some implementations, the series of acts 1400 is implemented in a digital environment for creating or editing digital content (e.g., digital images ). the act 1420 includes generating a grouping for each face in the second image that has a face descriptor that is within a similarity threshold of a face descriptor of a face in the first image and determining the missing face in the second image based on the missing face not belonging to a grouping (e.g., a grouping of different faces in different images belonging to the same person). In some implementations, the act 1420 includes determining the missing person in the second image by determining that the face descriptors corresponding to the faces detected in the first image are above (e.g., exceed or do not satisfy) a face similarity threshold when compared to the face descriptor for the missing person in the second image.).
Regarding claim 9, Lin et al. discloses The electronic device of claim 1, wherein the first group of individuals is one of a plurality of groups of individuals and to identify the first group of individuals among the plurality of groups of individuals, the least one processor:
processes the at least one first individual identified within the first preliminary image through an artificial intelligence engine ( Para 49-57; the face detection model 204 is a machine-learning model. For context, a machine-learning model can include a computer representation that can be tuned (e.g., trained) based on inputs to approximate unknown functions .) , which associates the at least one first individual identified within the first preliminary image with at least one of the plurality of groups of individuals (Para 51-57; For example, the image merging system can identify a person's face based on identifying visual element related to a user's face or head, such as a user's eyes, nose, mouth, hair, forehead, eyebrows, ears, hat, or glasses. ) ;
assigns each of the at least one first individual identified within the first preliminary image to at least one of the plurality of groups of individuals (Para 57-70; face descriptors; the image merging system can group matching faces across the two (or more) images by person. For example, the image merging system assigns faces within the similarity threshold (e.g., matching faces) to their own group. the image merging system analyzes the images in the set to determine if a threshold number of people or objects match across the images. For example, if at least one face matches across the base image and the second image, the image merging system can validate the second image. ) ; and identifies the first group of individuals from among the assigned groups of individuals based on an analysis by the artificial intelligence engine (Para 49-57; the face detection model 204 is a machine-learning model. For context, a machine-learning model can include a computer representation that can be tuned (e.g., trained) based on inputs to approximate unknown functions . the face detection model 204 is a neural network. Often, a neural network refers to a machine learning model that includes interconnected artificial neurons that communicate and learn to approximate complex functions and generate outputs based on a plurality of inputs provided to the model.) of which group a majority of the individuals are associated with to within a threshold certainty ( Para 57-70; he image merging system can group matching faces across the two (or more) images by person. For example, the image merging system assigns faces within the similarity threshold (e.g., matching faces) to their own group. In these implementations, faces in the second image that are not assigned to a group can correspond to a person missing from the base image. Further, in some implementations, the image merging system prevents more than one person from an image from being assigned to the same group. For example, if two faces in the second image are each within the similarity threshold of a face in the base image, the image merging system assigns the face in the second image having the higher similarity score (e.g., shorter L2 similarity distance) with the face in the base image to a group.).
Regarding claims 11-19 , the subject matter of claims 11-19 are similar to the subject matter disclosed in clams 1-9 respectively; therefore, claims 11-19 are rejected for the same reasons as set forth in claims 1-9 respectively.
Regarding claim 20, the subject matter of claim 20 is similar to the subject matter disclosed in clam 1; therefore, claim 20 is rejected for the same reasons as set forth in claim 1.
2. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US Pub No.: US 2021/0272253 A1), in view of Cofer et al. (US Pub. No.: US 2002/0125435 A1), and Lee et al. (US Pub. No.: US 2016/0094700 A1).
Regarding claim 10, Lin discloses The electronic device of claim 1, further comprising: at least one sensor communicatively coupled to the at least one processor ( Para 178; Fig. 15; input/output (“I/O”) interfaces 1508, ) .
However, Lin et al. does not disclose at least one sensor communicatively coupled to the at least one processor and which enables identification of a primary user of the electronic device; wherein the at least one processor:
receives a first sensor output from the at least one sensor; determines if the first sensor output is substantially similar to a reference sensor output that corresponds to an identity of the primary user of the electronic device; and in response to the first sensor output not being substantially similar to the reference sensor output, disables the missing image mode to prevent a non-primary user of the electronic device from integrating the image of the missing at least one second individual to the first image.
Lee et al. discloses at least one sensor ( Para 172; a touch screen or button including a fingerprint sensor ) communicatively coupled to the at least one processor ( Para 325; the electronic device 101 may be operations that are performed under the control of the processor 120) and which enables identification of a primary user (Para 172; acquire user information (e.g., fingerprint information) from an input means) of the electronic device; wherein the at least one processor:
receives a first sensor output from the at least one sensor ( Para 172; acquire user information (e.g., fingerprint information) from an input means (e.g., a finger) for entering an input on the display 150 ) ; determines if the first sensor output is substantially similar to a reference sensor output that corresponds to an identity of the primary user of the electronic device ( Para 172; The first electronic device 101 may identify a specific user on the basis of the acquired user information (e.g., may determine whether the acquired user information is consistent with preset information through a comparison) ; and in response to the first sensor output not being substantially similar to the reference sensor output, disables certain camera mode to prevent a non-primary user of the electronic device to have full access of the camera functions ( Para 172-175; in instances in which the message received from the second electronic device 102 is for changing the first electronic device 101 to a specific mode (e.g., a “kids” mode), the first electronic device 101 may change to the “kids” mode. The first electronic device 101 may restrict a predetermined function from being used in response to the change to the “kids” mode. Wherein since the camera is changed to kids mode; that means the user is not the primary user/adult who can be authorized to have no restriction in utilizing the device).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Lin et al. and Cofer with the teaching of Lee et al. in order to compare user input with preset information to determine whether the current user can have access to the image editing mode to merge images so that the primary user’s privacy can be protected.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XI WANG whose telephone number is (469)295-9155. The examiner can normally be reached on 9:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached on 571-272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XI WANG/Primary Examiner, Art Unit 2637
Prosecution Insights