DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 10/08/2025 have been fully considered but they are not persuasive.
Regarding claim 1, Applicant alleges that Iyer does not describe 2-step process as disclosed by limitations “detecting a time gap between the first and the second mappings and detecting at least a portion of mapping of a second object, from the plurality of objects, overlapping at least a portion of the first or the second mapping of the first object; and based at least in part on the detection: generating a new content structure comprising the first object, wherein the first and second mappings of the first object are concatenated to a) remove the time gap and b) exclude the portion of mapping of the second object.” After further reviewing Iyer reference, the Examiner respectfully disagrees. Iyer discloses (¶0004-¶0005, ¶0024-¶0025, ¶0027-¶0029) identifying objects in frames and such objects are identified in non-contiguous frames; as each frame has a sequence, duration, and time, lack of presence of the identified object signifies time gaps between such frames/mappings; (¶0039) frames that includes secondary object are given lower weight and discarded; (¶0029, ¶0041-¶0042) the content with primary objects are concatenated/summarized by retaining/combining these frames that include the primary object of interest in the summarized content while the rest of the frames devoid of the primary object or included with the secondary object are discarded; (¶0004-¶0005, ¶0024-¶0025, ¶0027-¶0029, ¶0056) the plurality of frames devoid of the primary object of interest or reference of the primary objects are discarded to maintain continuity of the presentation of frames containing identified object and keeping only frames that include primary objects, and therefore disclosing the new content segment does not comprise a time gap. When discarding frames that does not include primary object as taught by Iyer is similar to time gap that does not include first object. Therefore, it moots Applicant’s argument and the rejection is maintained.
Regarding claim 3, Applicant asserts that Park does not disclose “wherein generating the new content structure further comprises: retrieving one or more environmental templates from a data structure, wherein the one or more environmental templates comprise an environmental content structure, separate from the new content structure; retrieving one or more objects from the environmental content structure; and adding the one or more objects from the environmental content structure to the new content structure.” Examiner respectfully disagrees. Combination of Iyer and Park teaches this limitation, where Iyer discloses (¶0041-¶0042) that the content with primary objects are concatenated/summarized, and Park discloses (¶0120-¶0128) synthesizing the object layer with the background frame in the process of providing the summarized video, where background frame is extracted where the device synthesizes the object layer with the background frame; car is used as environmental template and the people are used as the objects as represented in Fig. 5; (¶0024-¶0025) summarized video and the background frame obtained from the image stream are separate. Together, they render obviousness of the claim. The rejection is maintained.
Regarding claim 5, Applicant alleges that Block does not disclose “rendering the vector data structure on a display, wherein the rendered vector data represents the first object.” Examiner respectfully disagrees. Block discloses (¶0014-¶0018) that the vector feature spaces may be assigned feature vector values which are algorithmically developed values reflecting characteristics of a particular feature type determined for an object within a captured visual scene; assessment of feature vector spaces for features of objects, such as people, may include frontal images, side images, other images, sound signatures, colors, sizes, affinities to other objects, as well as other features that may be attributed to an object. Therefore, it moots Applicant’s argument and the rejection is maintained.
Furthermore, the Double Patenting rejection is maintained since the arguments presented in Remarks section does not overcome cited reference(s) of the record.
See the rejection below.
Double Patenting
Claims 1-5, 8, 11-15, and 18 stand rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3 and 7-9 of U.S. Patent No. 11,330,307. Although the claims at issue are not identical, they are not patentably distinct from each other because the instant application is an obvious variation, recited similarly to the patented subject matter as shown in the Office Action mailed on 06/17/2025.
Claims 6-7 and 16-17 stand rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3 and 7-9 of US Patent 11,330,307 in view of US PG Pub 2019/0163982 to Block as shown in the Office Action mailed on 06/17/2025.
Claims 9-10 and 19-20 stand rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3 and 7-9 of US Patent 11,330,307 in view of US PG Pub 2015/0310642 to Kruglick as shown in the Office Action mailed on 06/17/2025.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 8, 11-12, and 18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US PG Pub 2020/0311434 to Iyer (“Iyer”).
Regarding claim 1, “A method comprising: accessing a video comprising a plurality of segments, wherein a first segment, from the plurality of segments, comprises depiction of a plurality of objects” reads on the method/system for summarizing multimedia content that includes plurality of objects (abstract) disclosed by Iyer and represented in Fig. 3. Iyer further discloses (¶0044-¶0045) that the device receives multimedia content comprising scenes.
As to “selecting a first object, from the plurality of objects in the first content segment” Iyer discloses (¶0023-¶0025, ¶0037-¶0041) that the device identifies the primary object from the plurality of objects from the multimedia content where the primary object is selected based on the parameter associated with the primary object such as frequency of appearance of the primary object, duration of appearance of the primary object in the multimedia content.
As to “determining that the selected first object includes first and second mappings indicating first and second time periods when the first object appears in the first content segment” Iyer discloses (¶0038-¶0042) that the system determines corresponding times associated with the primary object in the multimedia content as shown in TABLE 2 AND 3.
As to “detecting a time gap between the first and the second mappings and detecting at least a portion of mapping of a second object, from the plurality of objects, overlapping at least a portion of the first or the second mapping of the first object” Iyer discloses (¶0004-¶0005, ¶0024-¶0025, ¶0027-¶0029) identifying objects in frames and such objects are identified in non-contiguous frames; as each frame has a sequence, duration, and time, lack of presence of the identified object signifies time gaps between such frames/mappings; (¶0039) frames that includes secondary object are given lower weight and discarded.
As to “based at least in part on the detection: generating a new content structure comprising the first object, wherein the first and second mappings of the first object are concatenated to a) remove the time gap and b) exclude the portion of mapping of the second object” Iyer discloses (¶0041-¶0042) that the content with primary objects are concatenated/summarized; (¶0004-¶0005, ¶0024-¶0025, ¶0027-¶0029, ¶0056) the plurality of frames devoid of the primary object of interest or reference of the primary objects are discarded to maintain continuity of the presentation of frames containing identified object and keeping only frames that include primary objects, and therefore disclosing the new content segment does not comprise a time gap.
Regarding claim 2, “The method of claim 1, further comprising: generating a new content segment based on the new content structure; and transmitting the new content segment to an electronic device” Iyer discloses (¶0047, ¶0041-¶0042) that the summarized content is generated that includes identified object and the summarized multimedia content is rendered on the user device.
Regarding claim 8, “The method of claim 1, wherein the concatenation of the first and second mappings of the first object results in one continuous mapping from the first mapping to the second mapping of the first object without any gaps in-between” Iyer discloses (¶0041-¶0042) that the content with primary objects are concatenated/summarized; (¶0004-¶0005, ¶0024-¶0025, ¶0027-¶0029) the plurality of frames devoid of the primary object of interest or reference of the primary objects are discarded to maintain continuity of the presentation of frames containing identified object, and therefore disclosing the new content segment does not comprise a time gap.
Regarding claim 11, see rejection similar to claim 1.
Regarding claim 12, see rejection similar to claim 2.
Regarding claim 18, see rejection similar to claim 8.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Iyer in view of US PG Pub 2019/0251364 to Park (“Park”).
Regarding claim 3, Iyer meets all the limitations of the claim except “The method of claim 1, wherein generating the new content structure further comprises: retrieving one or more environmental templates from a data structure, wherein the one or more environmental templates comprise an environmental content structure, separate from the new content structure; retrieving one or more objects from the environmental content structure; and adding the one or more objects from the environmental content structure to the new content structure.” However, Park discloses (¶0120-¶0128) synthesizing the object layer with the background frame in the process of providing the summarized video where background frame is extracted where the device synthesizes the object layer with the background frame; car is used as environmental template and the people are used as the objects as represented in Fig. 5; (¶0024-¶0025) summarized video and the background frame obtained from the image stream are separate.
Therefore, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the invention to modify Iyer’s system by adding objects to retrieved environmental template as taught by Park in order to generate a summarized video based on the object and the background frame by eliminating the unwanted objects and focus only on objects of interest without distraction (Park - ¶0007).
Regarding claim 13, see rejection similar to claim 3.
Claims 4-7 and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Iyer in view of US PG Pub 2019/0163982 to Block (“Block”).
Regarding claim 4, “The method of claim 1, wherein the selecting the first object is based on the first object appearing most from among the plurality of objects in the first content segment” Iyer discloses (¶0023-¶0025, ¶0037-¶0041) that the device identifies the primary object from the plurality of objects from the multimedia content where the primary object is selected based on the parameter associated with the primary object such as frequency of appearance of the primary object, duration of appearance of the primary object in the multimedia content.
As to “the method further comprising: identifying the first object, from the plurality of objects in the first content segment, as the object appearing the most from among the plurality of objects in the first content segment” Iyer discloses (¶0023-¶0025, ¶0037-¶0041) that the device identifies the primary object from the plurality of objects from the multimedia content where the primary object is selected based on the parameter associated with the primary object such as frequency of appearance of the primary object, duration of appearance of the primary object in the multimedia content.
Iyer meets all the limitations of the claim except “generating a vector data structure for the identified first object.” However, Block discloses (¶0014-¶0018) that the vector feature spaces may be assigned feature vector values which are algorithmically developed values reflecting characteristics of a particular feature type determined for an object within a captured visual scene; assessment of feature vector spaces for features of objects, such as people, may include frontal images, side images, other images, sound signatures, colors, sizes, affinities to other objects, as well as other features that may be attributed to an object. Therefore, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the invention to modify Iyer’s system by generating vector data for the objects as taught by Block in order to identify/label objects with high degree of accuracy/confidence while utilizing well known mathematical vector/matrix calculations (Block - ¶0020).
Regarding claim 5, “The method of claim 4, further comprising, rendering the vector data structure on a display, wherein the rendered vector data represents the first object” Block discloses (¶0014-¶0018) that the vector feature spaces may be assigned feature vector values which are algorithmically developed values reflecting characteristics of a particular feature type determined for an object within a captured visual scene; assessment of feature vector spaces for features of objects, such as people, may include frontal images, side images, other images, sound signatures, colors, sizes, affinities to other objects, as well as other features that may be attributed to an object.
Regarding claim 6, “The method of claim 4, wherein the vector data structure includes data defining interconnected vectors or polygon” Block discloses (¶0018) that vector feature spaces may be assigned feature vector values which are algorithmically developed values reflecting characteristics of a particular feature type determined for an object within a captured visual scene. For example, a feature may be frontal image of a person and aspects of the frontal image such as spacing, size, colors, relative locations and other aspects of facial features for the person in an image view captured for a visual scene. The observed feature may have a multi-dimensional feature vector value generated for it which may be used to correlate it to other instances of captured frontal images of people or objects previously observed.
Regarding claim 7, “The method of claim 4, wherein the identifying of the first object is performed by using image recognition” Block discloses (¶0051, ¶0055) that the scene feature extractor such as image recognition system applies recognition algorithms to identify types of objects within the instance of observation for a captured scene of interest.
Regarding claim 14, see rejection similar to claim 4.
Regarding claim 15, see rejection similar to claim 5.
Regarding claim 16, see rejection similar to claim 6.
Regarding claim 17, see rejection similar to claim 7.
Claims 9-10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Iyer in view of US PG Pub 2015/0310642 to Kruglick (“Kruglick”).
Regarding claim 9, Iyer meets all the limitations of the claim except “The method of claim 1, further comprising, generating a new content segment based on the new content structure by reconstructing the first object in a frame-by-frame manner.” However, Kruglick discloses (¶0020) that the system combines a plurality of graphic assets (objects) into one data structure where rasterization includes taking graphic assets described in a vector graphical format and converting the graphic assets into a raster image for output on a display device; (¶0046-¶0047) the system generates the atlas from graphic assets where the atlas is a partitioned graphics formatted for consumption by the client device. Therefore, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the invention to modify Iyer’s system by reconstructing first object in the frame as taught by Kruglick in order to support and capture complex details of the image with a high degree of realism.
Regarding claim 10, “The method of claim 9, wherein the reconstructing of the first object is performed by converting vectors associated with the first object to raster images” Kruglick discloses (¶0020, ¶0027) that the system combines a plurality of graphic assets (objects) into one data structure where rasterization includes taking graphic assets described in a vector graphical format and converting the graphic assets into a raster image for output on a display device.
Regarding claim 19, see rejection similar to claim 9.
Regarding claim 20, see rejection similar to claim 10.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINKAL R CHOKSHI whose telephone number is (571)270-3317. The examiner can normally be reached Monday - Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BRIAN T PENDLETON can be reached at (571)272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PINKAL R CHOKSHI/Primary Examiner, Art Unit 2425
Prosecution Insights