Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-14 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. (U.S. Patent Application Publication No. 2018/0241961), referred herein as Choi, in view of Yang et al. (U.S. Patent Application Publication No. 2021/0074002), referred herein as Yang.
Regarding claim 1, Choi teaches an information processing device comprising: circuitry configured to (fig 1A, processing device 100, circuitry 180)
perform control to analyze an imaged image acquired from a plurality of imaging devices that image a target space (figs 8-11 and 16; paragraphs 219 and 220; paragraph 221, lines 1-6; images obtained from imaging devices are analyzed),
determine one or more subjects as clipping targets from the imaged image (paragraph 222, lines 1-9; paragraph 223, the last 11 lines; particular target subjects are identified), and
perform clipping of a determined subject (paragraph 225, lines 1-4; paragraph 226; the target subjects are clipped),
wherein the imaged image includes a plurality of imaged images having partially overlapping view angles acquired from the plurality of imaging devices (fig 16, partially overlapping view angles 1651-1654 acquired from a plurality of imaging devices; paragraph 267; paragraph 268, lines 1-7; paragraph 270), and
wherein the circuitry determines the one or more subjects as clipping targets according to a recognized motion of each subject recognized based on motion in each region including the one or more subjects compared to motion in other regions of the imaged image (paragraph 215; paragraphs 228 and 230; paragraph 270; one or more clipping targets are determined according to motion recognition of each subject in particular regions of the image).
In order to recognizing movement, a certain amount of change must occur in the position of the object(s) compared to other regions in the image (such as static regions); thus one may infer that Choi teaches recognizing an amount of movement of subjects relative to other regions. Nevertheless, Choi does not explicitly teach recognizing motion of each subject based on an amount of motion in each region relative to amounts of motion in other regions.
However, in a similar field of endeavor, Yang teaches an information processing device configured to analyze an image of a target space acquired by an imaging device, determine one or more subjects as clipping targets from the image, and determining the one or more subjects as clipping targets according to recognized motion of each subject (figs 2-5; paragraph 16, lines 1-12; paragraph 22), and further configured to recognize motion of each subject based on an amount of motion in each region relative to amounts of motion in other regions (figs 6-8; paragraph 16, the last 18 lines; paragraph 23, lines 1-16; paragraphs 25 and 27).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine to motion amount clipping processing of Yang with the motion clipping processing of Choi because this helps improve the image processing by increasing both the efficiency and accuracy of the motion recognition (see, for example, Yang, paragraph 22, the last 9 lines; paragraph 27, the last 5 lines; paragraph 33, the last 5 lines).
Regarding claim 2, Choi in view of Yang teaches the information processing device according to claim 1, wherein the circuitry performs clipping in a range including at least a face of the determined subject (Choi, paragraph 222, lines 1-6; paragraph 223, the last 7 lines).
Regarding claim 3, Choi in view of Yang teaches the information processing device according to claim 2, wherein the circuitry preferentially determines each subject that satisfies a predetermined condition as a clipping target (Choi, paragraph 222, lines 6-9; paragraph 235, lines 3-13; Yang, paragraph 21; the motivation to combine is similar to that discussed above in the rejection of claim 1; Choi and Yang both teach this step, but it is noted that as claimed, this step is preferred, but not required – and by extension, claims 4-6 would also be preferred, not required).
Regarding claim 4, Choi in view of Yang teaches the information processing device according to claim 3, wherein the circuitry determines a singing subject as a clipping target as a subject that satisfies the predetermined condition (Choi, paragraph 221, lines 1-6; paragraph 222, lines 1-9).
Regarding claim 5, Choi in view of Yang teaches the information processing device according to claim 3, wherein the circuitry determines a subject positioned in a region of interest as a clipping target as a subject that satisfies the predetermined condition (Choi, figs 8-11; Yang, figs 2-8; paragraphs 21 and 27; the motivation to combine is similar to that discussed above in the rejection of claim 1; paragraph 222, lines 1-9; paragraph 223, the last 7 lines).
Regarding claim 6, Choi in view of Yang teaches the information processing device according to claim 3, wherein the circuitry determines a subject positioned at a center on a stage that is the target space as a clipping target as a subject that satisfies the predetermined condition (Choi, fig 8; paragraph 222, lines 1-9; paragraph 235, the last 9 lines; paragraph 237, lines 1-4).
Regarding claim 7, Choi in view of Yang teaches the information processing device according to claim 1, wherein the circuitry determines a fixed position on a stage as a clipping target in a case where a number of subjects is less than a predetermined number of clippings (Choi, figs 8 and 14; paragraph 221, lines 1-6; paragraph 222, lines 1-9; paragraph 258; paragraph 259, the last 10 lines; paragraph 260; figure 14 shows three clippings 1450 and one subject in 1410, which results in fixed position clipping target 1430).
Regarding claim 8, Choi in view of Yang teaches the information processing device according to claim 2, wherein the circuitry performs clipping by a number of clippings corresponding to a number of outputs of an image (Choi, figs 8, 10, 11, and 14; paragraph 223, the last 7 lines; paragraph 238; paragraph 243; paragraph 259, lines 1-8; Yang, figs 2-8; paragraphs 21 and 27; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 9, Choi in view of Yang teaches the information processing device according to claim 2, wherein the circuitry performs clipping in a range including one subject or clipping in a range including a plurality of subjects (Choi, figs 8, 10, and 11; paragraph 222, lines 1-9; paragraph 235; paragraph 242, lines 1-7; Yang, figs 2-8; paragraphs 21 and 27; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 10, Choi in view of Yang teaches the information processing device according to claim 9, wherein the circuitry performs clipping in a range including a predetermined margin above an uppermost part of a body of a subject as a clipping target (Choi, figs 8 and 11; paragraphs 223 and 224; paragraph 225, lines 1-4; paragraph 226; paragraph 242, lines 1-12).
Regarding claim 11, Choi in view of Yang teaches the information processing device according to claim 10, wherein the circuitry further performs clipping in a range including a margin in an eye line direction in a case where an eye line of a subject as the clipping target is directed to left and right (Choi, figs 8 and 11; paragraph 221; paragraph 222, lines 1-9; paragraph 242, lines 1-10; in figure 8 the subject faces right, in figure 11, the subjects face left, and in both cases, clipping includes a margin in the eye direction; alternatively, in another interpretation, paragraphs 103-105 describe left and right eye images to be used in the clipping process).
Regarding claim 12, Choi in view of Yang teaches the information processing device according to claim 2, wherein the circuitry further performs clipping in a range including at least a hand of the subject (Choi, figs 8, 10, and 14; paragraph 222, lines 1-9; paragraph 235; paragraph 259, lines 1-8; each shows clipping including at least a hand; Yang, figs 2-8; paragraphs 21 and 27; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 13, Choi in view of Yang teaches the information processing device according to claim 1, wherein the circuitry performs clipping in a range including a plurality of subjects and clipping in a range including one subject included in the plurality of subjects (Choi, figs 8, 10, and 11; paragraph 221; paragraph 222, lines 1-9; paragraph 235; paragraph 237, lines 1-7; paragraph 242, lines 1-10).
Regarding claim 14, Choi in view of Yang teaches the information processing device according to claim 1, wherein the circuitry performs clipping in a range including one or more subjects in a predetermined area on a stage (Choi, figs 8 and 11; paragraph 221; paragraph 222, lines 1-9; paragraph 242, lines 1-10; paragraph 243).
Regarding claims 18 and 19, the limitations of each of these claims substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 20, Choi in view of Yang teaches the information processing device according to claim 1, wherein the circuitry further determines the one or more subjects as clipping targets according to the recognized motion of each subject recognized based on the amount of motion in each region including the one or more subjects relative to amounts of motion in regions of the imaged image including one or more other subject not determined as clipping targets (Choi, paragraph 179; paragraph 215; paragraphs 228 and 230; paragraph 270; recognized motion compared to regions including non-clipped subjects in background, for example; Yang, figs 6-8; paragraph 16, lines 1-12 and the last 18 lines; paragraphs 21 and 22; paragraph 23, lines 1-16; paragraphs 25 and 27; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Choi, in view of Yang, and further in view of Baudisch et al. (“An Exploration of User Interface Designs for Real-Time Panoramic Photography”), May 2006, referred herein as Baudisch.
Regarding claim 15, Choi in view of Yang teaches the information processing device according to claim 1, wherein the plurality of imaging devices are disposed on a seat side of a stage, and wherein the circuitry arranges and displays the plurality of imaged images side by side (Choi, fig 8; fig 16, view angles 1651-1654 and side by side result 1630; fig 14, side by side images 1450; paragraph 258; paragraph 259, lines 1-11; paragraph 267; paragraph 268, lines 1-7; images of the singer are necessarily captured from cameras “on a seat side” of the stage, because any other camera position [i.e. side-stage or back-stage] would not result in a frontal view of the singer).
Choi in view of Yang does not explicitly teach that the circuitry displays the images in a partially overlapping state, and receives adjustment of an overlapping position.
However, in a similar field of endeavor, Baudisch teaches an information processing device comprising circuitry that acquires images of a target space and determines one or more areas as clipping target (fig 3; abstract, lines 1-2 and 6-14; page 153, the last two paragraphs under “Panoramic Viewfinder”), wherein the circuitry displays the images in a partially overlapping state, and receives adjustment of an overlapping position (figs 4 and 5b; pages 154-155, steps 3-5; page 159, the paragraph beginning “After…” and the paragraph beginning “Once…”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the overlap adjustments of Baudisch with the multi-camera image processing of Choi in view of Yang because this enables users to quickly and intuitively stitch the multiple view of the target space together in real-time, while increasing the efficiency and quality of the output image (see, for example, Baudisch, abstract, the last 10 lines; page 155, the paragraph beginning “This…”; paragraph 163, the paragraph beginning “At the expense…”).
Regarding claim 16, Choi in view of Yang, further in view of Baudisch teaches the information processing device according to claim 15, wherein the circuitry performs control for outputting a plurality of clipped images obtained by clipping to a device that switches a distribution image, and performs control for displaying the plurality of clipped images on a display unit together with the plurality of imaged images arranged side by side (Choi, fig 14; paragraph 257; paragraph 258, lines 1-6; paragraphs 259 and 260).
Regarding claim 17, Choi in view of Yang, further in view of Baudisch teaches the information processing device according to claim 15, wherein in a case where a subject as the clipping target moves from a first imaged image to a second imaged image of the plurality of imaged images arranged side by side, the circuitry switches an imaged image of a clipping source at a portion where the first imaged image and the second imaged image overlap (Choi, fig 16; paragraph 268 and 270; paragraph 271, the last 7 lines; paragraph 273).
Response to Arguments
Applicant’s arguments with respect to the prior art rejections have been fully considered, but are moot in view of the new ground(s) of rejection presented above. The Examiner agrees that the previously cited art alone does not teach the newly amended claim limitations; however, the Examiner respectfully submits that Choi in view of Yang teaches these limitations.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Satoh (U.S. Patent Application Publication No. 2015/0123990); Image processing device, image processing method, image processing program, and storage medium.
Chen (U.S. Patent Application Publication No. 2022/0321788); Video processing method and electronic device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights