DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1–24 are pending in this application.
Claim Objections
Claim 23 is objected to because of the following informalities:
Claim 23 line 1 “The system or claim 18” should read “The system of claim 18” . Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 12 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 12 recites the limitation " the minimum and maximum depth range " in line 3 and “the new depth and position information” in lines 6 and 7. There is insufficient antecedent basis for this limitation in the claim. For purposes of examination the limitations will be read as “a minimum and maximum depth range” and “a new depth and position information”.
Dependent claim 13 is rejected for the same reasons.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., abstract idea - mathematical calculation and mental process) without significantly more.
Step (1) Are the claims directed to a process, machine, manufacture, or composition of matter;
Step (2A) Prong One: Are the claims directed to a judicially recognized exception, i.e., a law of nature, a natural phenomenon, or an abstract idea;
Prong Two: If the claims are directed to a judicial exception under Prong One, then is the judicial exception integrated into a practical application;
Step (2B) If the claims are directed to a judicial exception and do not integrate the judicial exception, do the claims provide an inventive concept.
Step 1:
Claim 1 recites a method. Therefore, the claim is directed to the statutory categories of process.
Step 2A:
Prong One:
Claim 1 recites:
“partitioning…the image into a plurality of image segments, wherein the one or more subjects and one or more obstructions are represented as separate image segments of the plurality of image segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation encompasses a mental process of partitioning the image by separating the subjects and the obstructions which is practically capable of being performed in the human mind with the assistance of pen and paper.
“obtaining…depth information for the plurality of image segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Obtaining the depth information is simply computing the depth of the image segments which is practically capable of being performed in the human mind with the assistance of pen and paper.
“identifying one or more focal image segments of the plurality of image segments based on the depth information of the plurality of image segments.”. Under its broadest reasonable interpretation in light of the specification, the limitation encompasses a mental process of identifying the focal image segment based on the depth which is practically capable of being performed in the human mind with the assistance of pen and paper.
Prong Two:
This judicial exception is not integrated into a practical application. The additional elements of “by a processor” and “receiving, by a processor, an image including one or more subjects and one or more obstructions;” and “modifying the image based on the one or more focal image segments to generate a modified image.” amount to no more than mere necessary data gathering and applying because, under its broadest reasonable interpretation, it is simply using generic hardware to perform the abstract idea. Thus, they are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claims are thus directed to the abstract idea.
Step (2B):
Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The limitations “by a processor” and “receiving, by a processor, an image including one or more subjects and one or more obstructions and “modifying the image based on the one or more focal image segments to generate a modified image.” amount to no more than mere data gathering with general purpose hardware and provide no inventive concept. These elements, individually and in combination, are well-understood, routine, conventional activity. As such, the claim is ineligible.
Step 1:
Claims 2, 5-10, and 14 recite a method. Claims 16-24 recite an apparatus. Therefore, the claims are directed to the statutory categories of process and machine, respectively.
Step 2A:
Prong One:
Claims 2, 5-10, 14 and 16-24 merely narrow the previously recited abstract idea limitations. For the reasons described above, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mathematical calculation and mental process that are practically capable of being performed in the human mind with the assistance of pen and paper.
Prong Two:
These judicial exceptions are not integrated into a practical application nor includes additional elements that are sufficient to amount to significantly more. Thus, the claims are ineligible.
Step 1:
Claims 3, 4, and 11-13, recite a method. Therefore, the claims are directed to the statutory categories of process.
Step 2A:
Prong One:
Claim 3 recites:
“determining a segment score based on the depth information, a segment size, and a segment position”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Determining a segment score is simply computing the score based on depth information, a segment size, and a segment position which is practically capable of being performed in the human mind with the assistance of pen and paper.
Claim 4 recites:
“the depth information comprises an average depth of the image segment”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. The depth information comprising an average depth is simply computing the average depth of the image segment which is practically capable of being performed in the human mind with the assistance of pen and paper.
Claim 11 recites:
“assigning an initial focal cluster of segments, wherein the initial focal cluster of segments includes the focal point segment and other segments that are within a threshold depth and distance from the focal point segment and have an average depth and position score above a threshold average depth and position score.”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Assigning an initial focal cluster of segments is simply computing a threshold depth and distance; and computing an average depth and position score above a threshold average depth and position score which is practically capable of being performed in the human mind with the assistance of pen and paper.
Claim 12 recites:
“identifying segments that are within a threshold distance from the focal cluster of segments and within the minimum and maximum depth range of the focal cluster of segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Assigning a focal cluster of segments is simply computing a threshold distance within a minimum and maximum depth range which is practically capable of being performed in the human mind with the assistance of pen and paper.
Claim 13 recites:
“assigning a positional threshold in the image”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Assigning a positional threshold is simply computing a positional threshold which is practically capable of being performed in the human mind with the assistance of pen and paper.
Prong Two:
This judicial exception is not integrated into a practical application. The additional elements of “wherein identifying one or more focal image segments further comprises” and “the perspective of a capture viewpoint of the image.” and “assigning the focal cluster based on the depth and position of segments relative to the focal points segment comprises” and “appending the found segments to the focal cluster of segments; and repeating the identification and addition of segments with the new depth and position information of the focal cluster of segments until no segments are found within the threshold distance and depth range.” amount to no more than mere necessary data gathering and applying because, under its broadest reasonable interpretation, it is simply using generic hardware to perform the abstract idea. Thus, they are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claims are thus directed to the abstract idea.
Step (2B):
Claim 3, 4, and 11- 13 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The limitations “wherein identifying one or more focal image segments further comprises” and “the perspective of a capture viewpoint of the image.” and “assigning the focal cluster based on the depth and position of segments relative to the focal points segment comprises” and “appending the found segments to the focal cluster of segments; and repeating the identification and addition of segments with the new depth and position information of the focal cluster of segments until no segments are found within the threshold distance and depth range.” amount to no more than mere data gathering with general purpose hardware and provide no inventive concept. These elements, individually and in combination, are well-understood, routine, conventional activity. As such, the claim is ineligible.
Step 1:
Claim 15 recites an apparatus. Therefore, the claim is directed to the statutory categories of machine.
Step 2A:
Prong One:
Claim 15 recites:
“partition the image into a plurality of image segments, wherein the one or more subjects and one or more obstructions are represented as separate image segments of the plurality of image segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation encompasses a mental process of partition the image by separating the subjects and the obstructions which is practically capable of being performed in the human mind with the assistance of pen and paper.
“obtain depth information for the plurality of image segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation recites a mathematical calculation and falls into the mathematical concepts grouping of abstract ideas. Obtaining the depth information is simply computing the depth of the image segments which is practically capable of being performed in the human mind with the assistance of pen and paper.
“identify one or more focal image segments of the plurality of image segments based on the depth information of the plurality of image segments;”. Under its broadest reasonable interpretation in light of the specification, the limitation encompasses a mental process of identifying the focal image segment based on the depth which is practically capable of being performed in the human mind with the assistance of pen and paper.
Prong Two:
This judicial exception is not integrated into a practical application. The additional elements of “An image editing system for automatic detection and editing of image obstructions” and “an image data storage comprising a plurality of images;” and “a processor configured to modify the plurality of images, wherein the processor is configured to” and “receive an image including one or more subjects and one or more obstructions;” and “modify the image based on the one or more focal image segments to generate a modified image.” amount to no more than mere necessary data gathering and applying because, under its broadest reasonable interpretation, it is simply using generic hardware to perform the abstract idea. Thus, they are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application and the claims are thus directed to the abstract idea.
Step (2B):
Claim 15 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The limitations “An image editing system for automatic detection and editing of image obstructions” and “an image data storage comprising a plurality of images;” and “a processor configured to modify the plurality of images, wherein the processor is configured to” and “receive an image including one or more subjects and one or more obstructions;” and “modify the image based on the one or more focal image segments to generate a modified image.” amount to no more than mere data gathering with general purpose hardware and provide no inventive concept. These elements, individually and in combination, are well-understood, routine, conventional activity. As such, the claim is ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 14, 15-17 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Jangid et al (US 2024/0312251 A1) (hereinafter, Jangid).
Regarding claim 1, Jangid discloses a computer implemented method comprising (Paragraph [0005] “a method is provided for generating an image.”; Paragraph [0008-0009] ” an apparatus for generating an image is provided…one or more of the apparatuses described above is, is part of, or includes… personal computer, a laptop computer, a server computer,”):
receiving, by a processor, an image including one or more subjects and one or more obstructions (Paragraph [0032] “a captured image may include a subject-of-interest (e.g., a person) and unwanted subjects (e.g., one or more other people or objects in the background of the image).”);
partitioning, by the processor, the image into a plurality of image segments , wherein the one or more subjects and one or more obstructions are represented as separate image segments of the plurality of image segments (Paragraph [0066] “subject-detection/segmentation model 404 can use the classifications to segment the image into different portions associated with the one or more classifications. For example, the subject-detection/segmentation model 404 can segment the one or more images into different portions associated with different classifications, such as people, buildings, cars, furniture, plants, or the like…subject detection/segmentation model 404 may be specifically trained to segment images into portions representing subjects and portions not representing subjects”); Paragraph [0117] “pixels of image 1202 may be associated with subjects, for example, based on segmentation of image 1202 into subject segments and background segments by subject detection and/or segmentation model”);
obtaining, by the processor, depth information for the plurality of image segments (Paragraph [0064] “the depth sensors 412 can produce depth images that include depth values corresponding to pixel locations in one or more images captured by the image sensors”);
identifying one or more focal image segments of the plurality of image segments based on the depth information of the plurality of image segments (Paragraph [0068] “depth estimates may be, or may include, determining a distance between a subject and a camera which captured an image of the subject. Depth-estimation model 408 may determine depth estimates for one or more (or all) subjects identified by subject-detection/segmentation model 404. Depth-estimation model 408 may determine depth estimates based on a degree of focus of the subjects among other things.”; Paragraph [0069] “Subject-of-interest identifier 414 may identify the subjects-of-interest based on one or more criteria including, for example, gaze and/or depth.”); and
modifying the image based on the one or more focal image segments to generate a modified image (Paragraph [0035] “the systems and techniques may modify an image, for example, by removing unwanted subjects (e.g., people) from the image.”; Paragraph [0059] “the components of the image-modification system 400 include one or more image sensors 402, a subject-detection/subject-detection/segmentation model 404… a depth-estimation model 408, …depth sensors 412, a subject-of-interest identifier 414, and/or an image-recognition model 416.”).
Regarding claim 2, which claim 1 is incorporated, Jangid discloses wherein partitioning the image into a plurality of image segments further comprises classifying the plurality of image segments into two or more categories (Paragraph [0066] “subject-detection/segmentation model 404 can use the classifications to segment the image into different portions associated with the one or more classifications. For example, the subject-detection/segmentation model 404 can segment the one or more images into different portions associated with different classifications, such as people, buildings, cars, furniture, plants, or the like…subject detection/segmentation model 404 may be specifically trained to segment images into portions representing subjects and portions not representing subjects”; Paragraph [0117] “pixels of image 1202 may be associated with subjects, for example, based on segmentation of image 1202 into subject segments and background segments by subject detection and/or segmentation model”).
Regarding claim 14, which claim 1 is incorporated, Jangid discloses wherein the plurality of image segments comprise one or more focal image segments (Paragraph [0069] “Subject-of-interest identifier 414 may be, or may include, one or more models and/or algorithms for identifying a subject-of-interest of an image. Subject-of-interest identifier 414 may identify the subjects-of-interest based on one or more criteria including, for example, gaze and/or depth… Additionally or alternatively, subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest”) and one or more peripheral segments (Paragraph [0032] “a captured image may include a subject-of-interest (e.g., a person) and unwanted subjects (e.g., one or more other people or objects in the background of the image).”; Paragraph [0070] “subject-of-interest identifier 414 may determine a depth of a subject of interest relative to the camera and depth(s) of one or more subjects in the image relative to the camera. The subject-of-interest identifier 414 may identify any subject having a depth that is greater than (or outside of) a depth threshold difference from the depth of the subject-of-interest as an unwanted subject.”) and modifying the image based on the focal image segments comprises removing the peripheral segments from the image (Paragraph [0033] “FIG. 2 illustrates an example of an image 200 that is a result of modifying the image 100 of FIG. 1 to remove people 104 that are not the subjects-of-interest of image 100.”; Paragraph [0040] ”The systems and techniques may remove the unwanted subjects from the image. In some aspects, removing unwanted subjects may include replacing the removed subjects with background pixels. In some cases, the systems and techniques may replicate background pixels that are in the image to replace the pixels representing the removed subjects.”).
Regarding claim 15, Jangid discloses an image editing system for automatic detection and editing of image obstructions (Paragraph [0059] “The image-modification system 400 includes various components that are used to modify one or more images, such as removing unwanted subjects from the one or more images. The image-modification system 400 may identify one or subjects-of-interest and one or more unwanted subjects in the one or more images and may remove the unwanted subjects from the one or more images. “), comprising:
an image data storage comprising a plurality of images (image processor 350 may store image frames and/or processed images in random access memory (RAM) 140/3225, read-only memory (ROM) 145/1520, a cache, a memory unit, another storage device, or some combination thereof.); and
a processor configured to modify the plurality of images (an apparatus for generating an image is provided that includes at least one memory and at least one processor (e.g., configured in circuitry) coupled to the at least one memory.), wherein the processor is configured to:
receive an image including one or more subjects and one or more obstructions (Paragraph [0032] “a captured image may include a subject-of-interest (e.g., a person) and unwanted subjects (e.g., one or more other people or objects in the background of the image).”);
partition the image into a plurality of image segments, wherein the one or more subjects and one or more obstructions are represented as separate image segments of the plurality of image segments (Paragraph [0066] “subject-detection/segmentation model 404 can use the classifications to segment the image into different portions associated with the one or more classifications. For example, the subject-detection/segmentation model 404 can segment the one or more images into different portions associated with different classifications, such as people, buildings, cars, furniture, plants, or the like…subject detection/segmentation model 404 may be specifically trained to segment images into portions representing subjects and portions not representing subjects”; Paragraph [0117] “pixels of image 1202 may be associated with subjects, for example, based on segmentation of image 1202 into subject segments and background segments by subject detection and/or segmentation model”);
obtain depth information for the plurality of image segments (Paragraph [0064] “the depth sensors 412 can produce depth images that include depth values corresponding to pixel locations in one or more images captured by the image sensors”);
identify one or more focal image segments of the plurality of image segments based on the depth information of the plurality of image segments (Paragraph [0068] “depth estimates may be, or may include, determining a distance between a subject and a camera which captured an image of the subject. Depth-estimation model 408 may determine depth estimates for one or more (or all) subjects identified by subject-detection/segmentation model 404. Depth-estimation model 408 may determine depth estimates based on a degree of focus of the subjects among other things.”; Paragraph [0069] “Subject-of-interest identifier 414 may identify the subjects-of-interest based on one or more criteria including, for example, gaze and/or depth.”); and
modify the image based on the one or more focal image segments to generate a modified image (Paragraph [0035] “the systems and techniques may modify an image, for example, by removing unwanted subjects (e.g., people) from the image.”; Paragraph [0059] “the components of the image-modification system 400 include one or more image sensors 402, a subject-detection/subject-detection/segmentation model 404… a depth-estimation model 408, …depth sensors 412, a subject-of-interest identifier 414, and/or an image-recognition model 416.”).
Regarding claim 16 (drawn to a system), claim 16 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 16, and all the other limitations similar to claim 16 are not repeated herein, but incorporated by reference.
Regarding claim 17, which claim 15 is incorporated, Jangid discloses wherein the obtaining depth information for the plurality of image segments includes obtaining the average depth value of the one or more segments in the image (Paragraph [0037] “systems and techniques may determine a depth estimate relative to a subject-of-interest and compare the depth estimate relative to the subject-of-interest to the depth estimates relative to other subjects in the image. The systems and techniques may determine that subjects that are less than (or within) a depth threshold difference from the depth estimate of the subject-of-interest are subjects-of-interest. For example, having identified a subject-of-interest, the systems and techniques may determine that a second subject that is substantially the same distance from (e.g., the same distance plus or minus 10%) the camera which captured the image”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-13 and 18-24 are rejected under 35 U.S.C. 103 as being unpatentable over Jangid et al (US 2024/0312251 A1) (hereinafter, Jangid) in view of Sun et al. (US 10929655 B2) (hereinafter, Sun).
Regarding claim 3, which claim 1 is incorporated, Jangid discloses wherein identifying one or more focal image segments further comprises determining a segment score based on the depth information (The examiner interprets criteria in Jangid as segment score. Paragraph [0035] “The systems and techniques may determine which subjects to retain and which to remove based on one or more criteria, such as based on… depths of the one or more subjects, and/or other criteria.”), [a segment size], and a segment position of the plurality of image segments (Paragraph [0064] “depth sensors 412 can obtain measurements of distance corresponding to objects in a captured scene… depth sensors 412 can produce depth images that include depth values corresponding to pixel locations in one or more images captured by the image sensors”).
However, Jangid fails to teach a segment size.
Sun teaches a segment size (Column 17, lines 26-33 “the body scale of a body segment of a person displayed in the image 130, a scoring rule 299 may include a body scale scoring rule 299. The body scale scoring rule 299 may be a ratio as follows: (height of the body segment) (width of the body segment)/(height of the image 130) (width of the image 130). The body scale scoring rule 299 is a ratio between a size of the body segment and a size of the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective
filing date to modify Jangid’s reference to include a segment size taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 3.
Regarding claim 4, which claim 3 is incorporated, Jangid discloses wherein the depth information comprises an average depth of the image segment from the perspective of a capture viewpoint of the image (Paragraph [0037] “systems and techniques may determine a depth estimate relative to a subject-of-interest and compare the depth estimate relative to the subject-of-interest to the depth estimates relative to other subjects in the image. The systems and techniques may determine that subjects that are less than (or within) a depth threshold difference from the depth estimate of the subject-of-interest are subjects-of-interest. For example, having identified a subject-of-interest, the systems and techniques may determine that a second subject that is substantially the same distance from (e.g., the same distance plus or minus 10%) the camera which captured the image”).
Regarding claim 5, which claim 4 Is incorporated, Jangid discloses wherein a lower average depth of a segment contributes towards a higher segment score for the segment (Paragraph [0037] “systems and techniques may determine a depth estimate relative to a subject-of-interest and compare the depth estimate relative to the subject-of-interest to the depth estimates relative to other subjects in the image. The systems and techniques may determine that subjects that are less than (or within) a depth threshold difference from the depth estimate of the subject-of-interest are subjects-of-interest. For example, having identified a subject-of-interest, the systems and techniques may determine that a second subject that is substantially the same distance from (e.g., the same distance plus or minus 10%) the camera which captured the image”; Paragraph [0069] “subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest. For example, subject-of-interest identifier 414 may identify all subjects a substantially a same distance from a camera as a subject-of-interest as additional subjects-of-interest.“).
Regarding claim 6, which claim 3 is incorporated, Jangid fails to teach wherein a larger size of a segment relative to the size of the image contributes towards a higher segment score for the segment.
Sun teaches wherein a larger size of a segment relative to the size of the image contributes towards a higher segment score for the segment (Column 16, lines 47-50 “pre-defined scores 252 may be stored for one or more of the continuous number of attributes 251 that describe this region of interest, in which a higher pre-defined score 252 signals a higher aesthetic value for that region of interest.”; Column 17, lines 3-6 “a scoring rule 299 is a value that is computed and considered during the determination of a score 280 for a region of interest or attribute 251 of an image 130 being analyzed.”; Column 17, lines 26-33 “the body scale of a body segment of a person displayed in the image 130, a scoring rule 299 may include a body scale scoring rule 299. The body scale scoring rule 299 may be a ratio as follows: (height of the body segment) (width of the body segment)/(height of the image 130) (width of the image 130). The body scale scoring rule 299 is a ratio between a size of the body segment and a size of the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include wherein a larger size of a segment relative to the size of the image contributes towards a higher segment score for the segment taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 6.
Regarding claim 7, which claim 3 is incorporated, Jangid fails to teach wherein a closer proximity of a segment to a horizontal center of the image contributes towards a higher segment score for the segment.
Sun teaches wherein a closer proximity of a segment to a horizontal center of the image contributes towards a higher segment score for the segment (Column 17, lines 9-19 “for the positional attribute 260 describing the body location of a body segment of a person displayed in the image 130, a scoring rule 299 may include a body horizontal location scoring rule 299 and a body vertical location scoring rule 299. The body horizontal location scoring rule 299 may be a horizontal distance between a center of the body segment and a center of the image 130. The horizontal distance may be normalized against the width of the image 130 to determine a score 280 for the positional attribute 260 describing the body location of the body segment.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include teaches wherein a closer proximity of a segment to a horizontal center of the image contributes towards a higher segment score for the segment taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 7.
Regarding claim 8, which claim 3 is incorporated, Jangid discloses wherein the focal image segments include a focal cluster of at least two image segments (Paragraph [0066] “subject-detection/segmentation model 404 may be specifically trained to segment images into portions representing subjects and portions not representing subjects.”; Paragraph [0078] “the computing device (or the one or more components thereof) can detect one or more subjects in one or more of the images… The subjects may be detected using a subject-detection model…”; Paragraph [0079-0080] “Additionally or alternatively, subjects within a depth threshold from subjects-of-interest may additionally be identified as subjects-of-interest… another subject that has a depth that is greater than (or outside of) the depth threshold difference of a subject-of-interest may be identified as an unwanted subject regardless of whether the other subject is gazing into the camera or not. Additionally or alternatively, any subject that is not a subject-of-interest may be identified as an unwanted subject.”).
Regarding claim 9, which claim 8 is incorporated, Jangid discloses wherein the image segment with the highest segment score is assigned as a focal point segment (Paragraph [0069] “Subject-of-interest identifier 414 may be, or may include, one or more models and/or algorithms for identifying a subject-of-interest of an image. Subject-of-interest identifier 414 may identify the subjects-of-interest based on one or more criteria including, for example, gaze and/or depth… Additionally or alternatively, subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest”).
Regarding claim 10, which claim 9 is incorporated, Jangid discloses wherein assigning a focal cluster of segments further comprises assigning the focal cluster based on the depth and [position] of segments relative to the focal point segment (Paragraph [0037] “systems and techniques may determine depth estimates relative to the subjects. The depth estimates may represent distances between an image sensor that captured the image and the respective subjects. The systems and techniques may determine a depth estimate relative to a subject-of-interest and compare the depth estimate relative to the subject-of-interest to the depth estimates relative to other subjects in the image.”).
However, Jangid fails to teaches based on the position of the segments.
Sun teaches based on the position of the segments (Column 25, lines 7-12 “The positional attributes 260A and 260B may describe a positioning of the body segment 403, and thus, has a dependency with the region of interest 703A. The positional attribute 260A describes a position of the person within the image 130, and the positional attribute 260B describes a scale of the person relative to the size of the image 130.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include based on the position of the segments taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 10.
Regarding claim 11, which claim 10 is incorporated, Jangid discloses wherein assigning the focal cluster based on the depth and [position] of segments relative to the focal points segment comprises assigning an initial focal cluster of segments (Paragraph [0037] “systems and techniques may determine depth estimates relative to the subjects. The depth estimates may represent distances between an image sensor that captured the image and the respective subjects. The systems and techniques may determine a depth estimate relative to a subject-of-interest and compare the depth estimate relative to the subject-of-interest to the depth estimates relative to other subjects in the image.”),
wherein the initial focal cluster of segments includes the focal point segment and other segments that are within a threshold depth and distance from the focal point segment and have an average depth and [position score] above a threshold average depth and [position score] (Paragraph [0037] “ systems and techniques may determine that subjects that are less than (or within) a depth threshold difference from the depth estimate of the subject-of-interest are subjects-of-interest. For example, having identified a subject-of-interest, the systems and techniques may determine that a second subject that is substantially the same distance from (e.g., the same distance plus or minus 10%) the camera which captured the image as the subject-of-interest is also a subject-of-interest”).
However, Jangid fails to teach based on the position of segments and an average position score above a threshold average position score.
Sun teaches based on the position of segments (Column 25, lines 7-12 “The positional attributes 260A and 260B may describe a positioning of the body segment 403, and thus, has a dependency with the region of interest 703A. The positional attribute 260A describes a position of the person within the image 130, and the positional attribute 260B describes a scale of the person relative to the size of the image 130.”) and an average position score above a threshold average position score (Column 2, lines 24-29 “a respective score for each of the plurality of positional attributes based on the training data, wherein the training data further comprises a plurality of pre-defined scores for each of the plurality of positional attributes, and wherein the aggregated score is computed based on each of the respective scores of the plurality of positional attributes.”; Column 17, lines 3-11 “a scoring rule 299 is a value that is computed and considered during the determination of a score 280 for a region of interest or attribute 251 of an image 130 being analyzed. In some embodiments, each attribute 251 or region of interest may have corresponding scoring rules 299 are used in the determination of a score 280 for that attribute 251. As an example, for the positional attribute 260 describing the body location of a body segment of a person displayed in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include based on the position of segments and an average position score above a threshold average position score taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 11.
Regarding claim 12, which claim 11 is incorporated, Jangid discloses wherein assigning a focal cluster of segments further comprises: identifying segments that are within a threshold distance from the focal cluster of segments and within the minimum and maximum depth range of the focal cluster of segments (Paragraph [0037] “ systems and techniques may determine that subjects that are less than (or within) a depth threshold difference from the depth estimate of the subject-of-interest are subjects-of-interest. For example, having identified a subject-of-interest, the systems and techniques may determine that a second subject that is substantially the same distance from (e.g., the same distance plus or minus 10%) the camera which captured the image as the subject-of-interest is also a subject-of-interest”; Paragraph [0069] “subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest.”);
appending the found segments to the focal cluster of segments (Paragraph [0069] “subject-of-interest identifier 414 may identify all subjects a substantially a same distance from a camera as a subject-of-interest as additional subjects-of-interest. Thus, if a group of people are posing together for a photo, and one of the people is not looking into the camera, that person may be identified as a subject-of-interest based on that person being at substantially the same depth as the others in the group.”); and
repeating the identification and addition of segments with the new depth and [position information] of the focal cluster of segments until no segments are found within the threshold distance and depth range (Paragraph [0068] “Depth-estimation model 408 may be, or may include, a machine-learning model trained to determine a depth estimate of one or more subjects (e.g., people). Determining depth estimates may be, or may include, determining a distance between a subject and a camera which captured an image of the subject. Depth-estimation model 408 may determine depth estimates for one or more (or all) subjects identified by subject-detection/segmentation model 404.”; Paragraph [0069] “Additionally or alternatively, subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest.”).
However, Jangid fails to teach the new position information.
Sun teaches the new position information (Column 25, lines 7-12 “The positional attributes 260A and 260B may describe a positioning of the body segment 403, and thus, has a dependency with the region of interest 703A. The positional attribute 260A describes a position of the person within the image 130, and the positional attribute 260B describes a scale of the person relative to the size of the image 130.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include the new position information taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 12.
Regarding claim 13, which claim 12 is incorporated, Jangid fails to teach wherein assigning a focal cluster of segments further comprises assigning a positional threshold in the image, wherein segments positioned within the positional threshold are appended to the focal cluster of segments.
Sun teaches wherein assigning a focal cluster of segments further comprises assigning a positional threshold in the image, wherein segments positioned within the positional threshold are appended to the focal cluster of segments (Column 2, lines 24-29 “a respective score for each of the plurality of positional attributes based on the training data, wherein the training data further comprises a plurality of pre-defined scores for each of the plurality of positional attributes, and wherein the aggregated score is computed based on each of the respective scores of the plurality of positional attributes.”; Column 17, lines 3-11 “a scoring rule 299 is a value that is computed and considered during the determination of a score 280 for a region of interest or attribute 251 of an image 130 being analyzed. In some embodiments, each attribute 251 or region of interest may have corresponding scoring rules 299 are used in the determination of a score 280 for that attribute 251. As an example, for the positional attribute 260 describing the body location of a body segment of a person displayed in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include wherein assigning a focal cluster of segments further comprises assigning a positional threshold in the image, wherein segments positioned within the positional threshold are appended to the focal cluster of segments taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 13.
Regarding claim 18, which claim 15 is incorporated, Jangid discloses wherein identifying one or more focal image segments comprises (Paragraph [0032] “a captured image may include a subject-of-interest (e.g., a person) and unwanted subjects (e.g., one or more other people or objects in the background of the image).”):
generating segment scores for one or more segments in the image based on the average depth value (Paragraph [0035] “The systems and techniques may determine which subjects to retain and which to remove based on one or more criteria, such as based on… depths of the one or more subjects, and/or other criteria.”), [size], and position of the segment (Paragraph [0064] “depth sensors 412 can obtain measurements of distance corresponding to objects in a captured scene… depth sensors 412 can produce depth images that include depth values corresponding to pixel locations in one or more images captured by the image sensors”);
assigning a focal point segment of the image based on the segment scores of all segments in the image (Paragraph [0069] “Subject-of-interest identifier 414 may be, or may include, one or more models and/or algorithms for identifying a subject-of-interest of an image. Subject-of-interest identifier 414 may identify the subjects-of-interest based on one or more criteria including, for example, gaze and/or depth… Additionally or alternatively, subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest”; Paragraph [0068] “Depth-estimation model 408 may be, or may include, a machine-learning model trained to determine a depth estimate of one or more subjects (e.g., people). Determining depth estimates may be, or may include, determining a distance between a subject and a camera which captured an image of the subject. Depth-estimation model 408 may determine depth estimates for one or more (or all) subjects identified by subject-detection/segmentation model 404.”);
assigning a focal cluster of segments based on the depth and [position] of segments relative to the focal point segment (Paragraph [0069] “another subject that has a depth that is greater than (or outside of) the depth threshold difference of a subject-of-interest may be identified as an unwanted subject regardless of whether the other subject is gazing into the camera or not. Additionally or alternatively, any subject that is not a subject-of-interest may be identified as an unwanted subject.”);
appending segments to the focal cluster of segments based on the depth and [position] of segments relative to the focal cluster of segments (Paragraph [0069] “subject-of-interest identifier 414 may identify all subjects a substantially a same distance from a camera as a subject-of-interest as additional subjects-of-interest. Thus, if a group of people are posing together for a photo, and one of the people is not looking into the camera, that person may be identified as a subject-of-interest based on that person being at substantially the same depth as the others in the group.”; Paragraph [0069] “Additionally or alternatively, subject-of-interest identifier 414 may identify subjects-of-interest by identifying subjects within a depth threshold from a previously-identified subjects-of-interest.”).
However, Jangid fails to teach based on the position of segments and assigning a positional threshold in the image, wherein segments located within the positional threshold are appended to the focal cluster of segments.
Sun teaches based on the position of segments (Column 25, lines 7-12 “The positional attributes 260A and 260B may describe a positioning of the body segment 403, and thus, has a dependency with the region of interest 703A. The positional attribute 260A describes a position of the person within the image 130, and the positional attribute 260B describes a scale of the person relative to the size of the image 130.”) and assigning a positional threshold in the image, wherein segments located within the positional threshold are appended to the focal cluster of segments (Column 2, lines 24-29 “a respective score for each of the plurality of positional attributes based on the training data, wherein the training data further comprises a plurality of pre-defined scores for each of the plurality of positional attributes, and wherein the aggregated score is computed based on each of the respective scores of the plurality of positional attributes.”; Column 17, lines 3-11 “a scoring rule 299 is a value that is computed and considered during the determination of a score 280 for a region of interest or attribute 251 of an image 130 being analyzed. In some embodiments, each attribute 251 or region of interest may have corresponding scoring rules 299 are used in the determination of a score 280 for that attribute 251. As an example, for the positional attribute 260 describing the body location of a body segment of a person displayed in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Jangid’s reference to include based on the position of segments and assigning a positional threshold in the image, wherein segments located within the positional threshold are appended to the focal cluster of segments taught by Sun’s reference. The motivation for doing so would have been to describe the region of interest that has a higher score signaling a higher aesthetic value as suggested by Sun (see Sun [Column 16, lines 43-50]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Sun with Jangid to obtain the invention specified in claim 18.
Regarding claim 19 (drawn to a system), claim 19 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 19, and all the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
Regarding claim 20 (drawn to a system), claim 20 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 20, and all the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 21 (drawn to a system), claim 21 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 21, and all the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Regarding claim 22 (drawn to a system), claim 22 is rejected the same as claim 9 and the arguments similar to that presented above for claim 9 are equally applicable to the claim 22, and all the other limitations similar to claim 9 are not repeated herein, but incorporated by reference.
Regarding claim 23 (drawn to a system), claim 23 is rejected the same as claim 11 and the arguments similar to that presented above for claim 11 are equally applicable to the claim 23, and all the other limitations similar to claim 11 are not repeated herein, but incorporated by reference.
Regarding claim 24 (drawn to a system), claim 24 is rejected the same as claim 12 and the arguments similar to that presented above for claim 12 are equally applicable to the claim 24, and all the other limitations similar to claim 12 are not repeated herein, but incorporated by reference.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Vanchinathan et al. (US 2024/0422284 A1) discloses a system and method for processing image data by segmenting the image data into a foreground and background portion. The method further includes removing the foreground portion from the background portion.
Gori et al. (US 2024/0144520 A1) discloses a method and system for creating a segmented mesh of an image. The editing system is configured to identify and remove distracting objects from a digital image.
Liu et al. (US 2024/0013351 A1) discloses a system and method for identifying unwanted object from a segmented image and removing the unwanted object from the acquired image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UROOJ FATIMA whose telephone number is (571)272-2096. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UROOJ FATIMA/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676