DETAILED ACTION
Claims 1-43 are currently pending in the application. Claims 1-20 are original claims to patent US 9,998,667 B2 to Nichols et al. (herein Nichols ‘667) and claims 21-43 are newly added claims.
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/10/2025 has been entered.
Reissue Applications
For reissue applications filed before September 16, 2012, all references to 35 U.S.C. 251 and 37 CFR 1.172, 1.175, and 3.73 are to the law and rules in effect on September 15, 2012. Where specifically designated, these are “pre-AIA ” provisions.
For reissue applications filed on or after September 16, 2012, all references to 35 U.S.C. 251 and 37 CFR 1.172, 1.175, and 3.73 are to the current provisions.
Applicant is reminded of the continuing obligation under 37 CFR 1.178(b), to timely apprise the Office of any prior or concurrent proceeding in which Patent No. 9,998,667 is or was involved. These proceedings would include any trial before the Patent Trial and Appeal Board, interferences, reissues, reexaminations, supplemental examinations, and litigation.
Applicant is further reminded of the continuing obligation under 37 CFR 1.56, to timely apprise the Office of any information which is material to patentability of the claims under consideration in this reissue application.
These obligations rest with each individual associated with the filing and prosecution of this application for reissue. See also MPEP §§ 1404, 1442.01 and 1442.04.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claims 7, 13, 36, 38, and 40
Upon review of the originally filed disclosure and prosecution history of 16/900,652, the following is noted for sake of clarity.
Independent claim 7 recites, “… a non-transitory computer readable medium having a sequence of instructions for recording a video, which when executed by a processing unit of a system, causes the system to …”. This language does not recite the word “means”, and therefore creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 USC 112(f)/6th.
Independent claim 13 recites, “… a non-transitory computer readable medium having a sequence of instructions for recording a video, which when executed by the processing unit of the system, causes the system to …”. This language does not recite the word “means”, and therefore creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 USC 112(f)/6th.
Independent claim 36 (and by extension dependent claim 38) recites, “… a processing unit that executes processing instructions stored in memory, wherein execution of the instructions by the processing unit processes … the processing unit processes the set of video frames by …”. This language does not recite the word “means”, and therefore creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 USC 112(f)/6th.
Independent claim 40 recites, “… a non-transitory computer-readable medium, having embodied thereon a program executable by a processor to perform …”. This language does not recite the word “means”, and therefore creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 USC 112(f)/6th.
A review of the 9,998,667 patent disclosure does not reveal any cause to rebut the presumption. For example, a plain reading of the claim language shows “the sequence of instructions” have sufficient structure as they are embodied on a non-transitory computer readable medium and are executed by a processing unit or processor. Further, the claimed “processing unit” or “processor” have sufficient structure as it performs execution. These findings are supported by the disclosure (Nichols ‘667: 3:52-53).
As such, the presumption is not rebutted since the claim limitations recite function and sufficient structure, material or acts to entirely perform the recited function.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-31 is/are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over US 2011/0228112 A1 to Kaheel et al. (herein Kaheel) in view of US 2003/0152291 A1 to Stephen Cheatle (herein Cheatle) in further view of US 2005/0168583 A1 to Graham Thomason (herein Thomason).
Claim 21
Kaheel discloses a method of automated video editing, the method comprising:
receiving a set of video frames captured by a camera over a period of time, the camera
associated with a selected orientation and a set of bounds (see paragraph [0029] – a smart phone captures a video sequence; the camera is inherently associated with a set of bounds that defines the borders of each image captured by the camera; see also paragraph [0036] – the camera and captured video sequence is associated with a selected “reference orientation”); and
processing the set of video frames during capture by the camera over the period of time,
wherein processing the set of video frames comprises:
identifying when at least one of the set of video frames is captured at an angle of
cant relative to the selected orientation associated with the camera (see paragraphs [0036]-[0038] – video frames captured at an angle of cant relative to the reference orientation are identified and adjusted),
determining the angle of cant of the at least one identified video frame (see paragraphs [0029] and [0032]-[0033] – the angle of cant of each of the captured video frames is determined),
rotating the at least one identified video frame by the determined angle of cant,
wherein the at least one identified video frame is rotated towards the selected orientation associated with the camera (see paragraphs [0035]-[0038] – video frames captured at an angle of cant relative to the reference orientation are adjusted by rotating them to correspond to the reference orientation).
Kaheel does not appear to expressly disclose:
cropping the at least one rotated video frame in accordance with the set of bounds associated with the camera, wherein cropping the at least one rotated video frame in accordance with the set of bounds includes maximizing an area of one of the bounds and resizing the cropped video frame to have at least one of a same height and width as one or more other video frames in the set of video frames.
Cheatle discloses tilt correction of electronic images. In particular, Cheatle teaches correcting the tilt of an image by cropping a rotated version of the original image in accordance with a predefined set of image composition rules (see paragraph [0001]). FIG. 1 of Cheatle illustrates a conventional way of cropping a rotated image. FIGS. 1b, 1c, and 1d show possible locations of a bounding rectangle for cropping the rotated image. In FIGS 1b and 1c, the area in the crop rectangle is maximized to fit within the bounds of the rotated image – i.e., as shown in the figures, the cropped area represents the largest upright-oriented bounding rectangle that can fit within the rotated image. See paragraph [0010] – “Both the crop boundary 3 shown in FIG. 1b and the crop boundary 4 shown in FIG. 1c represent the largest possible rectangular crop boundary that will fit within the rotated image 1” (emphasis added). In FIG. 1d, the area in the crop rectangle is maximized to fit within the bounds of the rotated image when the crop rectangle is centered within the captured image. See paragraph [0010] – “FIG. 1d . . . shows the largest crop boundary that will fit within the original tilted image having its centre . . . coincident with that of the original image.”
Accordingly, Cheatle expressly discloses that it was conventional to set a crop boundary as the largest possible tilt-adjusted rectangle that fits within a captured image in one of at least three locations within the originally captured image. This is considered the same as, or at least equivalent to, “maximizing an area of one of the bounds,” as claimed.
It would have been obvious to those skilled in the art to modify the teachings of Kaheel by those of Cheatle to crop each rotated video frame to maximize a set of bounds, as claimed, since Cheatle establishes that it was conventional to crop images that have been subjected to tilt correction using a bounding rectangle in order to produce a maximal crop area, and those skilled in the art would have understood that the conventional cropping techniques described in relation to FIGS. 1b, 1c, and 1d of Cheatle are effective for their intended purposes of designating crop regions of tilt-corrected images..
Thomason discloses image rotation correction for video equipment. In particular, Thomason teaches performing a tilt correction of a captured video frame, such as illustrated in FIGS. 3b, 3c, and 3d. Upon producing the tilt-corrected frame, the frame is then resized to match the size of the originally captured frame. That is, the tilt-corrected frame of FIG. 3d is resized as shown in FIG. 3e, which matches the size of the originally captured frame illustrated in FIG. 3b. See also paragraph [0037] – the corrected image is resized so that the final image frame is the same size as the original image frame. In view of this teaching, those skilled in the art would have found it obvious to resize a tilt-corrected frame, as claimed, since resizing the resultant image provides the convenience of maintaining the same dimensions of a sequences of image frames so that the sequence can be played back at a consistent size.
Claim 22
Kaheel, Cheatle, and Thomason teaches the method of claim 21, further comprising storing the processed set of video frames in memory at an end of the period of time, wherein each video frame in the processed set corresponds to the selected orientation associated with the camera (see 220 at FIG. 2 of Kaheel and paragraphs [0048]-[0049] – after being subjected to rotation processing, the video frames are stored in memory; the video frames can be rotated in real-time so that the rotated versions of the video frames are stored).
Claim 23
Kaheel, Cheatle, and Thomason teaches the method of claim 21, further comprising storing the received set of video frames in memory, wherein each of the video frames in the received set is unedited (see 220 at FIG. 2 of Kaheel and paragraphs [0048]-[0049] – before being subjected to rotation processing, the video frames are stored in memory; the video frames can be rotated later in time so that the unedited versions of the video frames are stored).
Claim 24
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein the at least one identified video frame is processed automatically without user input in real-time upon capture of the at least one identified video frame, and wherein the at least one processed video frame is available for immediate sharing over one or more networks (see Kaheel at paragraphs [0048]-[0049] – the video frames can be rotated in real-time; see also FIG. 8 and paragraph [0084] – the mobile device 831 can transmit processed images to other devices and/or service providers; see also paragraph [0029] – the video images can be streamed from the smart phone to other destinations).
Claim 25
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein the set of bounds in the selected orientation is oriented without a cant relative to level ground (see paragraph [0036] of Kaheel – the reference orientation can be zero degrees of rotation, i.e., level with the ground).
Claim 26
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein determining the angle of cant is based on a magnitude of a tilt of the camera from the selected orientation (see paragraphs [0032]-[0033] of Kaheel – the angle of cant of each of the captured video frames is based on the magnitude of tilt in three dimensions; i.e., the angles of cant ( α and β ) are based on the magnitudes of tilt ( ax, ay, and az ) calculated by the accelerometer).
Claim 27
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein the magnitude of cant of the camera is indicated by accelerometer data from an accelerometer associated with the camera (see paragraphs [0032]-[0033] of Kaheel – the angle of cant of each of the captured video frames is based on the magnitude of tilt in three dimensions; i.e., the angles of cant ( α and β ) are based on the magnitudes of tilt ( ax, ay, and az ) calculated by the accelerometer).
Claim 28
Kaheel, Cheatle, and Thomason teaches the the method of claim 27, further comprising identifying a point during the period of time at which the at least one identified video frame was captured, and identifying the accelerometer data at the identified point (see paragraph [0029] of Kaheel – accelerometer data is identified for each video frame, and each video frame is inherently associated with a particular point in time in a video sequence).
Claim 29
Kaheel, Cheatle, and Thomason teaches the method of claim 27, further comprising receiving accelerometer data at a plurality of points during the period of time, wherein each of the plurality of points is associated with accelerometer data (see paragraph [0029] of Kaheel – accelerometer data is identified for each video frame, and each video frame is inherently associated with a particular point in time in a video sequence).
Claim 30
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein the area of one of the bounds correspond to a bounded rectangle, and wherein cropping the at least one rotated video frame includes fitting the bounded rectangle inside the at least one rotated image to maximize an area of the bounded rectangle (i.e., whereas Kaheel’s camera is inherently associated with a set of bounds that defined the boundaries of the captured video frames, Cheatle further teaches defining a bounding rectangle that fits within the rotated image in order to produce a cropped, rotated image that is “pleasing” – see, e.g., FIG. 6, in which a bounding rectangle is fitted inside the rotated image so as to produce the cropped image having a maximal area, as shown in FIG. 7).
Claim 31
Kaheel, Cheatle, and Thomason teaches the method of claim 21, wherein the at least one cropped video frame is resized to correspond to an original size of the at least one identified video frame as captured by the camera (see Thomason as FIG. 3e and paragraph [0037]).
Claims 32-35 is/are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over US 2011/0228112 A1 to Kaheel et al. (herein Kaheel) in view of US 2003/0152291 A1 to Stephen Cheatle (herein Cheatle) in view of US 2005/0168583 A1 to Graham Thomason (herein Thomason) in further view of US 8,199,207 B2 to Yoshiyuki Kato (herein Kato).
Claim 32
Kaheel, Cheatle, and Thomason teaches the method of claim 21, but does not appear to teach wherein processing the set of video frames includes comparing each video frame in the set to a cant threshold, and further comprising identifying that the angle of cant of at least one identified video frame is greater than the cant threshold, wherein rotating the at least one identified video frame is based on the identification that the angle of cant of at least one identified video frame is greater than the cant threshold.
Kato discloses a tilt correction method in which each video frame in a series of video frames is compared to a cant threshold. See S8, FIG. 2 – the tilt angle for each frame is compared to a threshold of 1 degree.
Kato further discloses identifying that the angle of cant of each video frame is greater than 1 degree. See S8, FIG. – “NO” determination identifies a video frame as having a tilt greater than or equal to 1 degree.
Kato then performs rotational correction on each video frame (S16) based on the identification (S8: “NO” determination) that the angle of cant is greater than the cant threshold of 1 degree.
It would have been obvious to those skilled in the art to achieve the claimed invention by comparing each video frame to a cant threshold and rotating the frames based on the threshold being exceeded since Kato shows that it was conventional to only correct for tilt when at least a minimum amount of tilt has been determined. Accordingly, frames having a negligible amount of tilt will not be corrected.
Claim 33
Kaheel, Cheatle, Thomason, Kato teaches the method of claim 32, wherein another one of the video frames in the set has an angle of cant that is less than the cant threshold, and further comprising determining to apply one or more available post-capture processing operations to the another video frame based on the angle of cant (see S8, FIG. 2 of Kato – a “YES” determination indicates that the angle of cant of the another image is more than 1 degree, and based on this determination, post-capture processing operations S15 and S16 are executed using the HK parameter set to zero degrees at S9).
Claim 34
Kaheel, Cheatle, Thomason, Kato teaches the method of claim 33, further comprising applying at least one of the available post-capture processing operations to the another video frame, wherein the applied post-capture processing operations adjusts the angle of cant of the other video frame (see S16, FIG. 2 of Kato – a post-capture processing operation of rotational correction is applied).
Claim 35
Kaheel, Cheatle, Thomason, Kato teaches the method of claim 33, further comprising selecting the cant threshold from a plurality of different available cant thresholds (see Kato at column 5:35-45 – the minimum correction angle is predetermined to be a threshold of 1 degree, and is necessarily selected in advance from among the plurality of all other possible minimum correction angles).
Withdrawn Objections and Rejections
The following objections/rejections from the Office Action of 09/10/2024 are withdrawn in view of the Patent Owner Response of 02/10/2025: (1) 35 USC 112(a)/1st, Written Description; and (2) 35 USC 112(b)/2nd for claims 7-20, 36-41, 43, and 44. Additionally, the means-plus-function claim interpretation analysis under the 35 USC 112(f)/6th is clarified.
35 USC 112(a)/1st, Written Description
The previous 35 USC 112(a)/1st rejection for lack of written description of claims 1-6, 21-31, 33, 34, 36-41, 43, 44 is withdrawn.
Independent claims 21, 36, and 40, as amended by the response of 02/10/2025, do not have the previously rejected limitation, “determining which one or more available post-capture processing operations to utilize on the one or more other video frames captured during a period of time associated with the angle of cant, wherein the determined available post-capture processing operation is utilized when the angle of cant is less than the cant threshold but rotates beyond a predefined threshold angle of rotation”. As such, this rejection is withdrawn.
Upon review, reasonable support is found in the originally filed disclosure for the calculations to determine the angle of cant of video frames. For example, the ‘667 patent describes using accelerometers to measure acceleration of a system in distinct dimensions (x, y, and z) to determine the orientation of the system (Nichols ‘667: 3:48-51). Further, the ‘667 patent describes using accelerometers to determine the magnitude of tilt of the image by determining the degrees of tilt of the device when capturing the image (Nichols ‘667: 4:44-52). A person of ordinary skill would understand the various calculations of using dimensions (x, y, z), acceleration, and degrees. Therefore, the disclosure reasonably supports the disclosure of the limitations previously labeled (Office Action of 09/10/2024) as functional phrases #3, #4, #6, #7, #9, #10, #11, #12, #13 and the corresponding phrases in the independent claims. As such, previous written description rejections for lack of proper support related to these phrases are withdrawn (including claims 1, 7, 13, 21, 36, and 40).
35 USC 112(b)/2nd, Indefinite
The previous 35 USC 112(b)/2nd rejection for indefiniteness of claims 7-20, 36-41, 43, and 44 is withdrawn. As discussed above in the Claim Interpretation section, the claim limitations are not interpreted under 35 USC 112(f)/6th. Therefore, the claim language does have sufficient structure and the scope of the claim language can be reasonably determined.
35 USC 112(f)/6th, Claim Interpretation
With regard to mean-plus-function limitations, the claim language is being interpreted under 35 USC 112(f)/6th as stated above.
Response to Arguments
Patent Owner’s arguments (06/26/2024) with respect to claim(s) 18-19, 21-27, 29-34, 36-38 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Allowable Subject Matter
Claims 1-20 and 36-43 are allowed.
The following is a statement of reasons for the indication of allowable subject matter: the cited prior art of record does not teach or reasonably suggest the combined limitations of the independent claims.
Correspondence Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM H WOOD whose telephone number is (571)272-3736. The examiner can normally be reached Monday-Friday 7am-3pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexander Kosowski can be reached at (571)272-3744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/William H. Wood/
Reexamination Specialist, Art Unit 3992
Conferees:
/RACHNA S DESAI/Reexamination Specialist, Art Unit 3992
/ALEXANDER J KOSOWSKI/Supervisory Patent Examiner, Art Unit 3992